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Highlights 

Highlights  of  GAO-07-860,  a  report  to 
the  Subcommittee  on  Readiness  and 
Management  Support,  Committee  on 
Armed  Services,  U.S.  Senate 


DOD  BUSINESS  TRANSFORMATION 

Lack  of  an  Integrated  Strategy  Puts  the 
Army’s  Asset  Visibility  System 
Investments  at  Risk 


Why  GAO  Did  This  Study 

The  Department  of  Defense  (DOD) 
established  a  goal  to  achieve  total 
asset  visibility  (TAV)  over  30  years 
ago,  but  to  date  it  has  been 
unsuccessful.  GAO  was  requested 
to  (1)  determine  whether  the  Army 
has  a  systems  strategy  for 
achieving  TAV,  (2)  determine  if  the 
Army’s  business  system  investment 
governance  structure  is  consistent 
with  DOD  guidance,  and 
(3)  evaluate  the  Army’s  effort  to 
correct  previously  reported 
problems  with  the  Logistics 
Modernization  Program  (LMP). 
GAO  obtained  an  understanding  of 
the  Army’s  efforts  to  achieve  TAV, 
oversee  and  manage  its  business 
system  investments,  and  address 
previously  reported  LMP  problems. 


What  GAO  Recommends 


GAO  makes  five  recommendations 
to  DOD  and  the  Army:  (1)  develop 
a  concept  of  operations  for  the 
Army;  (2)  develop  policies, 
procedures,  and  processes  to 
manage  investments  from  a 
portfolio  perspective;  (3)  establish 
an  independent  verification  and 
validation  function;  (4)  require  that 
any  future  General  Fund  Enterprise 
Business  System  (GFEBS) 
economic  analysis  is  prepared  in 
accordance  with  applicable 
policies;  and  (5)  direct  that  LMP 
use  an  independent  system  test 
team.  Overall,  DOD  concurred 
with  the  recommendations  and 
stated  that  it  will  work  diligently  to 
close  them. 


www.gao.gov/cgi-bin/getrpt7GAO-07-860. 

To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  the  link  above. 
For  more  information,  contact  McCoy 
Williams  at  (202)  512-9095  or  Keith  Rhodes 
at  (202)  512-6412. 


What  GAO  Found 

Supply  chain  management  has  been  on  GAO’s  high-risk  list  since  1990.  One 
area  that  has  contributed  to  this  long-standing  problem  has  been  DOD’s 
inability  to  maintain  control  and  accountability  over  hundreds  of  billions  of 
dollars  of  assets.  DOD  plans  to  improve  its  asset  management  through  its 
business  system  modernization.  In  this  regard,  GFEBS,  the  Global  Combat 
Support  System- Army  (GCSS-Army),  and  LMP  are  aimed  at  achieving  TAV 
within  the  Army.  The  Army  estimates  that  it  will  invest  approximately  $5 
billion  to  develop  and  implement  these  systems.  However,  this  investment  is 
being  made  without  a  clear  integrated  strategy. 

•  GFEBS,  GCSS-Army,  and  LMP  are  not  being  developed  in  the  context  of 
a  well-defined  Army-wide  enterprise  architecture.  As  a  result,  the  Army 
does  not  have  an  informed  basis  for  determining  if  these  systems  will  fit 
within  the  context  of  future  Army  business  operations  and  will 
efficiently  and  effectively  address  the  Army’s  long-standing  weaknesses 
associated  with  the  lack  of  asset  visibility. 

•  The  Army  lacks  a  concept  of  operations  that  would  describe,  at  a  high 
level,  (1)  how  the  three  business  systems  relate  to  each  other  in 
achieving  the  Army’s  TAV  goal,  and  (2)  how  information  flows  from  and 
through  these  systems.  Moreover,  GAO  found  that  the  Army's  lack  of  a 
concept  of  operations  has  contributed  to  its  failure  to  take  full  advantage 
of  business  process  reengineering  opportunities  that  are  available  when 
using  an  enterprise  resource  planning  solution. 

Without  these  key  foundational  elements,  the  Army  is  at  risk  of  investing 
about  $5  billion  in  business  systems  and  still  not  achieving  DOD’s  and  the 
Army’s  goal  of  TAV. 

Furthermore,  while  the  Army  has  established  a  governance  structure  that  is 
consistent  with  DOD  guidance,  its  processes  are  still  maturing.  The  Army’s 
governance  structure  is  designed  to  certify  and  review  individual  business 
systems  rather  than  to  evaluate  these  investments  from  a  portfolio 
perspective.  Such  a  perspective  permits  investments  to  be  viewed  in  a 
comprehensive  manner  to  help  ensure  that  the  organization’s  missions  and 
objectives  are  achieved.  GAO  also  found  that  the  Army  did  not  have  reliable 
processes  and  analyses,  such  as  an  independent  validation  and  verification 
function  or  economic  analyses,  to  support  its  oversight  of  individual 
business  systems.  Until  the  Army's  investment  processes  mature,  it  runs  the 
risk  of  investing  in  business  systems  that  do  not  provide  the  desired 
functionality  and  efficiency. 

Additionally,  LMP  continues  to  be  plagued  by  problems  that  have  beset  the 
system  since  its  implementation  in  July  2003.  LMP  continues  to  experience 
problems  with  accurately  recognizing  revenue  and  billing  customers,  which 
can,  in  part,  be  attributed  to  ineffective  system  testing. 
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Accountability  *  Integrity  *  Reliability 


United  States  Government  Accountability  Office 
Washington,  DC  20548 


July  27,  2007 

The  Honorable  Daniel  K.  Akaka 
Chairman 

The  Honorable  John  Ensign 
Ranking  Member 

Subcommittee  on  Readiness  and  Management  Support 
Committee  on  Armed  Services 
United  States  Senate 

The  Department  of  Defense  (DOD)  has  continually  struggled  to  achieve 
and  maintain  efficient  and  effective  management  over  the  hundreds  of 
billions  of  dollars  it  has  invested  in  tangible  assets,  including  inventory, 
supplies,  and  materials  (inventory  and  related  property)  and  property, 
plant,  and  equipment  (PP&E).1  DOD  was  responsible  for  almost  72  percent 
($697  billion)  of  the  total  $970  billion  reported  governmentwide  value  for 
these  assets,  as  of  September  30,  2006.2  The  nature  and  severity  of  DOD’s 
financial  and  business  management  system  deficiencies  impede  the  ability 
of  DOD  managers  to  receive  the  full  range  of  information  needed  to 
effectively  manage  day-to-day  operations.  Of  the  27  areas  on  GAO’s  high- 


Federal  Accounting  Standards  and  the  Joint  Financial  Management  Improvement  Program 
(JFMIP)  define  inventory,  supplies,  and  materials  as  consisting  of  three  subclassifications: 
(1)  inventory — tangible  personal  property  that  is  (a)  held  for  sale,  (b)  in  the  process  of 
production  for  sale,  or  (c)  to  be  consumed  in  the  production  of  goods  for  sale  or  in  the 
provision  of  services  for  a  fee;  (2)  operating  materials  and  supplies — tangible  personal 
property  to  be  consumed  in  normal  operations;  and  (3)  stockpile  materials — strategic  and 
critical  materials  held  due  to  statutory  requirements  for  use  in  national  defense, 
conservation,  or  national  emergencies.  Property,  plant,  and  equipment  is  defined  as 
tangible  assets  that  have  an  estimated  useful  life  of  2  or  more  years,  are  not  intended  for 
sale  in  the  ordinary  course  of  business,  and  are  intended  to  be  used  or  available  for  use  by 
the  entity. 

2The  reported  amounts  are  net  of  allowances  and  depreciation,  as  applicable. 
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risk  list,3  DOD  has  8  high-risk  areas  of  its  own4  and  shares  responsibility 
for  7  governmentwide  high-risk  areas.5 

Visibility  over  its  assets  has  been  a  DOD  concern  for  decades.  If  the 
information  contained  in  the  asset  accountability  systems  is  not  accurate, 
complete,  and  timely,  DOD’s  day-to-day  operations  could  be  adversely 
affected  by  investing  in  inventory,  for  example,  that  is  not  needed  to  meet 
current  needs.  When  this  occurs,  the  department  may  obligate  funds 
unnecessarily,  which  could  lead  to  not  having  sufficient  obligational 
authority  to  purchase  needed  items.  In  recognition  of  the  importance  of 
asset  accountability  to  successful  operations,  the  department  established  a 
goal  to  achieve  total  asset  visibility  (TAV)  over  30  years  ago.  DOD  defines 
TAV  as  the  ability  to  provide  timely  and  accurate  information  on  the 
location,  movement,  status,  and  identity  of  units,  personnel,  equipment, 
and  supplies  and  having  the  ability  to  act  on  that  information.  Over  the 
years,  the  military  services  and  defense  components  have  undertaken 
numerous  initiatives  to  achieve  TAV.  Within  the  Army,  one  such  initiative 
has  been  the  Logistics  Modernization  Program  (LMP).  In  May  2004  and 
June  2005,  we  reported  that  LMP  was  not  providing  the  Army  the  promised 
capability.6  DOD’s  current  estimate  for  achieving  TAV  is  2010. 

This  report  provides  information  in  support  of  your  continuing  oversight 
of  DOD’s  progress  towards  resolving  the  department’s  long-standing 
problems  in  achieving  TAV.  As  you  requested,  our  initial  effort  was 
directed  at  the  Army.  In  September  2006,  the  Army  reported  inventory  and 


3GAO,  High-Risk  Series:  An  Update,  GAO-07-310  (Washington,  D.C.:  January  2007). 

4The  eight  specific  DOD  high-risk  areas  are  (1)  business  transformation,  (2)  business 
systems  modernization,  (3)  contract  management,  (4)  financial  management,  (5)  personnel 
security  clearance  program,  (6)  supply  chain  management,  (7)  support  infrastructure 
management,  and  (8)  weapon  systems  acquisition. 

5The  seven  high-risk  areas  that  DOD  shares  responsibility  for  are  (1)  disability  programs, 
(2)  information  sharing  for  homeland  security,  (3)  information  systems  and  critical 
infrastructures,  (4)  interagency  contracting,  (5)  human  capital,  (6)  real  property,  and 
(7)  technologies  critical  to  national  security  interests. 

6GAO,  DOD  Business  Systems  Modernization:  Billions  Continue  to  Be  Invested  with 
Inadequate  Management  Oversight  and  Accountability,  GAO-04-615  (Washington,  D.C.: 
May  27,  2004)  and  Army  Depot  Maintenance:  Ineffective  Oversight  of  Depot  Maintenance 
Operations  and  System  Implementation  Efforts,  GAO-05-441  (Washington,  D.C.:  June  30, 
2005). 
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related  property  of  about  $57  billion  and  PP&E  over  $85  billion.7  The  Army 
has  identified  three  primary  system  initiatives  directed  at  achieving  TAV 
within  the  service:  (1)  LMP,  (2)  Global  Combat  Support  System-Army 
Field/Tactical8  (GCSS-Army),  and  (3)  General  Fund  Enterprise  Business 
System  (GFEBS).  Our  objectives  were  to  (1)  determine  whether  the  Army 
has  developed  a  business  system  strategy  for  achieving  TAV,  (2)  determine 
if  the  Army  has  effectively  implemented  a  governance  structure  to  oversee 
and  manage  its  business  system  investments  in  accordance  with  DOD 
guidance,  and  (3)  evaluate  the  extent  to  which  the  Army  has  made 
progress  in  correcting  the  previously  reported  problems  regarding  LMP’s 
implementation. 

To  address  the  first  objective,  we  met  with  Army  program  office  officials 
for  GFEBS,  GCSS-Army,  and  LMP  and  obtained  briefings  on  the  intended 
purpose  of  each  system.  In  addition,  we  conducted  walkthroughs  and 
reviewed  documentation  related  to  various  transactions  to  obtain  an 
understanding  of  how  the  systems  would  exchange  data  and  to  assess  how 
the  Army  intended  to  use  these  systems  individually  and  collectively  to 
achieve  TAV.  To  address  the  second  objective,  we  reviewed  guidance 
issued  by  DOD,  the  Army,  and  the  Business  Transformation  Agency  (BTA) 
related  to  investment  management.  We  also  obtained  an  understanding  of 
the  Army’s  business  system  investment  governance  structure  and  process 
for  ensuring  compliance  with  the  certification  and  annual  system  review 
processes  required  by  the  fiscal  year  2005  National  Defense  Authorization 
Act.9  To  address  the  third  objective,  we  interviewed  and  obtained  briefings 
from  LMP  program  management  office  officials  and  others,  and  reviewed 
and  analyzed  LMP  system  requirement  and  testing  documentation  to 
assess  the  extent  to  which  corrective  actions  had  been  taken  or  are 
planned  to  address  our  prior  recommendations.  We  determined  that  the 
documentation  Army  prepared  and  submitted  through  its  business  system 
governance  process  to  the  investment  review  boards  (IRBs)  and  the 
Defense  Business  Systems  Management  Committee  (DBSMC)  as  a  basis 


7As  of  September  30,  2006,  the  Army  reported  net  inventory  and  related  net  property  valued 
at  about  $39  billion  for  the  Army  general  fund  and  about  $18  billion  for  the  Army  working 
capital  fund.  For  the  same  period,  Army  also  reported  net  PP&E  valued  at  over  $84  billion 
for  the  Army  general  fund  and  over  $1  billion  for  the  Army  working  capital  fund. 

sField/Tactical  refers  to  Army  units  that  are  deployable  to  locations  around  the  world  such 
as  Iraq  or  Afghanistan. 

9Ronald  W.  Reagan  National  Defense  Authorization  Act  for  Fiscal  Year  2005,  Pub.  L.  No. 
108-375,  §  332,  118  Stat.  1811,  1851-56  (Oct.  28,  2004)  (codified,  in  part,  at  10  U.S.C.  §§  186, 
2222). 
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for  approving  individual  Army  business  system  investments  was 
sufficiently  reliable  for  our  purposes.  Our  work  was  performed  from  May 
2006  through  June  2007  in  accordance  with  U.  S.  generally  accepted 
government  auditing  standards.  Details  on  our  scope  and  methodology  are 
included  in  appendix  I.  We  requested  comments  on  a  draft  of  this  report 
from  the  Secretary  of  Defense  or  his  designee.  We  received  written 
comments  from  the  Deputy  Under  Secretary  of  Defense  (Business 
Transformation),  which  are  reprinted  in  appendix  II. 


Results  in  Brief 


The  Army’s  current  approach  for  developing  GFEBS,  GCSS-Army,  and 
LMP  lacks  several  elements  that  are  critical  to  the  successful 
implementation  of  integrated  business  systems,  such  as  an  Army-level 
enterprise  architecture  (EA),10  a  concept  of  operations,  and  a  portfolio- 
based  rather  than  individual-project-based  business  system  investment 
review  process.  Without  these  key  foundational  elements,  the  risk  that  the 
Army’s  efforts  to  achieve  TAV  will  not  be  successful  is  greatly  increased. 
While  the  Army’s  efforts  to  develop  these  systems  and  transform  its 
logistics  operations  may  result  in  incremental  improvements,  without 
these  three  essential  elements,  they  are  unlikely  to  achieve  the  efficiencies 
that  can  be  attained  through  an  integrated  business  system  solution. 
Instead,  if  the  Army  continues  on  its  current  path,  it  runs  the  risk  of 
investing  significant  resources  to  simply  automate  its  existing  inefficient 
business  processes  using  more  current  technology. 

As  it  now  stands,  the  Army  plans  to  invest  about  $5  billion  over  the  next 
several  years  to  develop  and  implement  GFEBS,  GCSS-Army,  and  LMP 
without  the  benefit  of  a  well-defined  Army  EA.  We  reported  in  August 
2006, 11  that  the  Army  was  in  the  initial  stages  of  developing  an  EA.  As  of 
May  2007,  this  was  still  the  case.12  A  well-defined  EA  is  an  essential  tool  for 
leveraging  information  technology  (IT)  in  the  transformation  of  business 
and  mission  operations.  Our  experience  with  federal  departments  and 


10An  enterprise  architecture  is  a  blueprint  for  organizational  change  defined  in  models  that 
describe  (in  both  business  and  technology  terms)  how  the  entity  operates  today  and  how  it 
intends  to  operate  in  the  future;  it  also  includes  a  plan  for  transitioning  to  this  future  state. 

nGAO,  Enterprise  Architecture:  Leadership  Remains  Key  to  Establishing  and  Leveraging 
Architectures  for  Organizational  Transformation,  GAO-06-831  (Washington,  D.C.: 

Aug.  14,  2006). 

12GAO,  DOD  Business  Systems  Modernization:  Progress  Continues  to  Be  Made  in 
Establishing  Corporate  Management  Controls,  but  Further  Steps  Are  Needed,  GAO-07-733 
(Washington,  D.C.:  May  14,  2007). 
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agencies  has  shown  that  attempting  to  modernize  systems  without  an  EA 
to  guide  and  constrain  investments  often  results  in  operations  and  systems 
that  are  duplicative,  not  well  integrated,  unnecessarily  costly  to  maintain 
and  interface,  and  ineffective  in  supporting  mission  goals.13  Moreover,  the 
development,  implementation,  and  maintenance  of  an  EA  are  widely 
recognized  as  hallmarks  of  successful  public  and  private  organizations, 
and  their  use  is  required  by  the  Clinger-Cohen  Act  of  199614  and  the  related 
guidance  from  the  Office  of  Management  and  Budget  (OMB).15 

In  addition  to  the  Army’s  lack  of  an  EA,  the  Army  also  lacked  a  concept  of 
operations,  which  outlines  the  Army’s  strategy  for  achieving  TAV, 
including  how  the  three  systems  it  has  identified  as  key  in  attaining  TAV 
will  interoperate.  A  concept  of  operations  would  provide  the  Army  a 
forum  for  interchange  among  stakeholders — such  as  oversight  entities, 
program  managers,  developers,  and  users — on  major  technical  and 
programmatic  issues  related  to  achieving  TAV.  Without  a  concept  of 
operations  for  achieving  TAV,  the  Army  is  hindered  in  its  ability  to  apply 
an  enterprise  view  in  (1)  making  decisions  as  to  how  GFEBS,  GCSS-Army, 
and  LMP  will  individually  and  collectively  enhance  the  Army’s  asset 
accountability,  including  providing  TAV;  and  (2)  determining  what 
changes  are  needed  in  its  related  business  processes.  Additionally,  the 
Army’s  inability  to  achieve  TAV  hinders  its  and  DOD’s  efforts  to  resolve 
the  long-standing  problems  associated  with  supply  chain  management, 
which  has  been  on  our  high-risk  list  since  1990.  Asset  visibility  is  one  of 
the  focus  issues  critical  to  successfully  addressing  this  high-risk  area. 
Further,  the  Army’s  lack  of  a  concept  of  operations  has  resulted  in  its 
failure  to  take  full  advantage  of  business  process  reengineering 


13See,  for  example,  GAO,  Information  Technology:  FBI  Is  Taking  Steps  to  Develop  an 
Enterprise  Architecture,  but  Much  Remains  to  Be  Accomplished,  GAO-05-363 
(Washington,  D.C.:  Sept.  9,  2005);  Homeland  Security:  Efforts  Under  Way  to  Develop 
Enterprise  Architecture,  but  Much  Work  Remains,  GAO-04-777  (Washington,  D.C.:  Aug.  6, 
2004);  DOD  Business  Systems  Modernization:  Lim  ited  Progress  in  Development  of 
Business  Enterprise  Architecture  and  Oversight  of  Information  Technology  Investments, 
GAO-04-731R  (Washington,  D.C.:  May  17,  2004);  Information  Technology:  Architecture 
Needed  to  Guide  NASA’s  Financial  Management  Modernization,  GAO-04-43  (Washington, 
D.C.:  Nov.  21,  2003);  and  DOD  Business  Systems  Modernization:  Important  Progress 
Made  to  Develop  Business  Enterprise  Architecture,  but  Much  Work  Remains, 

GAO-03-1018  (Washington,  D.C.:  Sept.  19,  2003). 

14Pub.  L.  No.  104-106,  div.  E,  tit.  LI,  §§  5122,  5125,  110  Stat.  186,  683,  685  (Feb.  10,  1996) 
(codified  at  40  U.S.C.  §§  11312,  11315). 

15OMB  Circular  No.  A-130,  Management  of  Federal  Information  Resources,  §  8(b)  (Nov.  28, 
2000). 


Page  5 


GAO-07-860  Army  Asset  Visibility 


opportunities  that  are  available  when  using  an  enterprise  resource 
planning  (ERP)16  solution.  Rather,  the  Army’s  existing  strategy  perpetuates 
some  of  the  cumbersome  and  ineffective  business  processes  that  are 
currently  used  in  its  existing  legacy  system  environment.  The  benefits  of 
an  ERP  solution  include  streamlining  of  business  processes  and 
elimination  of  data  redundancy. 

Furthermore,  while  the  Army  has  established  a  business  system 
investment  management  governance  structure  that  is  consistent  with  DOD 
guidance,  its  overall  investment  management  approach  is  still  maturing. 
Currently,  the  Army’s  investment  review  process  is  designed  to  ensure  the 
completion  of  certifications  and  annual  reviews  of  individual  business 
systems  rather  than  to  evaluate  business  system  investments  from  a 
portfolio  management  perspective.  A  portfolio-based  perspective  permits 
an  organization  to  view  its  business  system  investments  in  a 
comprehensive  manner  to  help  ensure  that  the  organization’s  missions, 
strategic  goals,  and  objectives  are  achieved.  Moreover,  we  found  that  the 
Army  did  not  have  reliable  processes  or  analyses,  such  as  independent 
verification  and  validation  functions  or  economic  analyses,  to  support  its 
oversight  of  program  management  office  efforts  to  develop  and  implement 
business  systems.  Until  the  Army  adopts  a  business  system  investment 
management  approach  that  provides  for  reviewing  groups  of  systems  and 
making  enterprise  decisions  regarding  how  these  groups  will  collectively 
interoperate  to  provide  a  desired  capability,  it  runs  the  risk  of  investing 
significant  resources  in  business  systems  that  do  not  provide  the  desired 
functionality  and  efficiency. 

LMP  continues  to  be  plagued  by  operational  problems  that  have  beset  the 
system  virtually  since  its  initial  implementation  in  July  2003.  While  from  a 
“big  picture”  perspective,  an  EA , concept  of  operations,  and  effective  IT 
portfolio  management  are  essential  elements  in  an  entity’s  efforts  to 
transform  its  operations,  it  is  equally  important  for  the  entity  to  have  the 
disciplined  processes  needed  to  actually  implement  individual  business 
systems  on  time,  within  budget,  and  with  the  promised  capability.  As  we 
have  previously  reported,  historically  DOD  has  had  difficulty  in 
accomplishing  this  goal,  and  LMP  has  been  no  exception.  As  of  September 
2006,  the  Army  reported  that  it  had  obligated  approximately  $462  million 


16An  ERP  solution  is  an  automated  system  using  commercial  off-the-shelf  (COTS)  software 
consisting  of  multiple,  integrated  functional  modules  that  perform  a  variety  of  business- 
related  tasks  such  as  payroll,  general  ledger  accounting,  and  supply  chain  management. 
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to  develop  and  implement  LMP.  In  May  2004  and  June  2005,  we  reported 
on  operational  issues  related  to  LMP,  for  example,  the  inability  to 
accurately  recognize  revenue  and  bill  customers — a  problem  that 
continues  today.  We  recommended,  and  the  Army  agreed,  that  the 
implementation  of  LMP  should  be  delayed  until  the  operational  problems 
we  identified  were  resolved.  While  the  Army  is  working  to  resolve  LMP 
operational  issues,  we  continue  to  have  concerns  about  the  adequacy  of 
LMP’s  system  testing  given  that  the  continuing  problems  with  LMP  can,  in 
part,  be  attributed  to  ineffective  and  nonindependent  system  testing.  Until 
an  effective  LMP  testing  process  is  implemented,  the  Army  will  have  little 
assurance  that  the  corrective  actions  it  takes  (1)  are  properly  developed, 
and  (2)  do  not  introduce  additional  defects  into  the  system. 

We  are  making  five  recommendations  to  the  Secretary  of  Defense  to 
improve  the  department’s  efforts  to  achieve  TAV  and  further  enhance  its 
efforts  to  improve  its  control  and  accountability  over  business  system 
investments.  More  specifically,  we  recommend  that  the  Secretary  of 
Defense  (1)  develop  a  concept  of  operations  for  the  Army;  (2)  develop 
policies,  procedures,  and  processes  to  manage  investments  from  a 
portfolio  perspective;  (3)  establish  an  independent  verification  and 
validation  function;  (4)  ensure  the  GFEBS  economic  analysis  update  is 
prepared  in  accordance  with  applicable  guidance;  and  (5)  direct  that  LMP 
use  an  independent  system  test  team. 

We  received  written  comments  on  a  draft  of  this  report  from  the  Deputy 
Under  Secretary  of  Defense  (Business  Transformation),  which  are 
reprinted  in  appendix  II.  Overall,  DOD  concurred  with  our 
recommendations  and  stated  that  it  would  work  diligently  to  implement 
them.  The  comments  included  two  sets  of  specific  responses  to  our 
recommendations — one  set  provided  by  BTA  and  another  provided  by  the 
Army.  In  its  comments,  the  Army  concurred  with  each  of  the 
recommendations.  BTA  stated  that  it  fully  agreed  with  our  observations. 
BTA,  though,  partially  concurred  with  all  the  recommendations  on  the 
basis  that  they  were  directed  jointly  to  the  Secretary  of  the  Army  and  the 
Director,  BTA.  BTA’s  comments  noted  that  it  has  neither  the  authority  nor 
the  responsibility  to  direct  the  actions  of  the  Army. 

We  appreciate  the  department’s  willingness  to  address  our 
recommendations.  With  regard  to  BTA’s  concern,  our  recommendations 
do  not  direct  BTA  to  oversee  or  direct  the  Army.  Rather,  the 
recommendations  stated  that  the  specific  actions  should  be  undertaken 
jointly  at  the  direction  of  the  Secretary  of  Defense.  We  continue  to  believe 
that  a  cooperative  and  effectively  coordinated  BTA  and  Army  approach  to 
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addressing  our  recommendations  is  most  likely  to  achieve  the 
fundamental  business  system  transformation  necessary  to  achieve  the 
department’s  TAV  objective. 

Background 

TAV  has  been  elusive  within  DOD.  Timely,  reliable  information  on  the 
location,  quantity,  and  status  of  the  department’s  tangible  assets  could 
significantly  improve  its  ability  to  more  efficiently  and  effectively  deliver 
needed  items  to  DOD  operating  forces  and  thereby  enhance  military 
readiness.  The  department  has  recognized  the  importance  of  achieving 

TAV  and  included  it  as  part  of  its  overall  business  transformation  initiative, 
which  includes  the  development  and  implementation  of  a  business 
enterprise  architecture  (BEA).  The  Joint  Chiefs  of  Staff  also  identified  TAV 
as  one  of  four  fundamental  changes  needed  to  transform  the  department’s 
logistics  operations.17 

One  of  the  primary  factors  contributing  to  DOD’s  inability  to  provide 
management  with  TAV  is  DOD’s  outdated  and  ineffective  management 
information  system  environment.18  The  Federal  Financial  Management 
Improvement  Act  (FFMLA)  of  199619  and  other  financial  management 
reform  legislation  have  emphasized  the  importance  of  improving  financial 
management,  which  necessarily  encompasses  proper  inventory 
management,  across  the  federal  government.  Built  upon  the  foundation 
laid  by  the  Chief  Financial  Officers  Act  of  1990, 20  FFMLA  emphasizes  the 
need  for  agencies  to  have  integrated  financial  management  systems  that 
can  generate  timely,  accurate,  and  useful  information  to  make  informed 
decisions  and  to  ensure  accountability  on  a  continuous  basis.21  Lacking 
such  critical  information,  government  leaders  will  not  be  positioned  to 
invest  resources,  reduce  costs,  oversee  programs,  and  hold  agency 

17 DOD  identified  the  following  four  fundamental  changes  in  logistics  practices  that  are 
needed  to  transform  its  logistics  operations:  (1)  customer  wait  time,  (2)  time-definite 
delivery,  (3)  TAV,  and  (4)  Web-based,  shared  data  environment. 

18GAO,  Defense  Inventory:  DOD  Could  Improve  Total  Asset  Visibility  Initiative  with 
Results  Act  Framework,  GAO/NSIAD-99-40  (Washington,  D.C.:  Apr.  12,  1999). 

19Pub.  L.  No.  104-208,  div.  A,  §  101(f),  title  VIII,  110  Stat.  3009,  3009-389  (Sept.  30,  1996). 

20Pub.  L.  No.  101-576,  104  Stat.  2838  (Nov.  15,  1990). 

“TfMIA  requires  the  24  Chief  Financial  Officers  Act  departments  and  agencies  to 
implement  and  maintain  financial  management  systems  that  comply  substantially  with 
(1)  federal  financial  management  systems  requirements,  (2)  applicable  federal  accounting 
standards,  and  (3)  the  U.S.  Government  Standard  General  Ledger  at  the  transaction  level. 
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managers  accountable  for  the  manner  in  which  government  programs  are 
operated. 


Army  Initiatives  Aimed  at  To  improve  control  and  accountability  over  its  assets,  the  Army  has 
Achieving  TAV  embarked  on  a  multisystem  integration  effort  that  is  intended  to  leverage 

commercial  ERP  software  and  processes.  This  integration  is  the  focus  of 
the  Single  Army  Logistics  Enterprise  (SALE)  initiative.  SALE  is  designed  to 
integrate  information  technology  requirements,  business  processes, 
business  rules,  and  data  from  the  Army’s  logistics,  financial,  and 
acquisition  transactions  for  planning  and  supporting  warfighting  logistics 
operations.  According  to  the  Army,  SALE  is  to  provide  a  coordinated  ERP 
solution  built  around  two  individual  logistics  system  development  efforts: 
(1)  LMP  and  (2)  GCSS-Army.  Under  the  SALE  vision,  GCSS-Army  and  LMP 
will  be  integrated  into  a  single  solution  to  provide  an  Army-wide  logistics 
environment  spanning  from  “the  factory  to  foxhole.”  On  the  financial  side, 
the  Army  is  developing  GFEBS  to  provide  financial  visibility  over  its 
assets.  Each  of  these  efforts  is  described  below. 

LMP.  In  February  1998,  the  U.S.  Army  Materiel  Command  began  an  ERP 
effort — LMP — to  replace  its  legacy  materiel  and  maintenance  management 
systems — the  Commodity  Command  Standard  System  and  the  Standard 
Depot  System — with  LMP.  The  Army  has  been  using  the  existing  legacy 
systems  for  over  30  years.  LMP  is  intended  to  transform  the  Army’s 
Working  Capital  Fund  logistics  operations  in  six  core  processes:  order 
fulfillment,  demand  and  supply  planning,  procurement,  asset  management, 
materiel  maintenance,  and  financial  management.  LMP  became 
operational  at  the  U.S.  Army  Communications  and  Electronics  Command, 
Fort  Monmouth,  New  Jersey,  and  Tobyhanna  Army  Depot,  Tobyhanna, 
Pennsylvania,  in  July  2003.  The  initial  deployment  of  LMP  consisted  of 
inventory  items  such  as  electronics;  electronic  repair  components;  and 
communications  and  intelligence  equipment  such  as  night  vision  goggles, 
electronic  components  such  as  circuit  boards,  and  certain  munitions  such 
as  guidance  systems  included  in  missiles.  Figure  1  shows  the  LMP  time 
line  as  of  March  2007. 
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Figure  1:  LMP  Time  Line 
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Source:  DOD,  LMP  program  office. 


The  Communications-Electronics  Life  Cycle  Management  Command,22 
Tobyhanna  Army  Depot,  and  the  Defense  Finance  and  Accounting  Service 
(DFAS)  are  the  primary  LMP  users.  When  LMP  is  fully  implemented,  its 
capacity  is  expected  to  include  more  than  17,000  users  at  149  locations 
and  it  will  be  populated  with  6  million  Army-managed  inventory  items 
valued  at  about  $40  billion.  LMP  is  scheduled  to  reach  full  operational 
capability  (FOC)23  in  fiscal  year  2010.  As  of  September  30,  2006,  the  Army 
reported  that  approximately  $462  million  had  been  obligated  for  this 
system  effort  and  estimates  that  it  will  invest  at  least  another  $896  million 
in  LMP.24 

GCSS-Army.  The  GCSS-Army  program  was  initiated  in  1997  to  overcome 
duplicative  databases,  poor  asset  visibility,  and  stovepiped 
communications  between  numerous  existing  Army  logistics  systems.  The 
goal  of  GCSS-Army  is  to  integrate  multiple  logistic  functions  by  replacing 
numerous  legacy  systems  and  interfaces.  Since  the  program’s  inception,  it 
has  undergone  several  revisions,  including  a  change  from  a  custom 


22The  name  of  the  command  changed  from  Communications  and  Electronics  Command  to 
the  Communications-Electronics  Life  Cycle  Management  Command  in  February  2005. 

23Full  operational  capability  means  that  the  system  has  been  deployed  to  all  intended 
locations. 

24The  contractor  has  submitted  claims  totaling  $850  million  to  the  Army.  According  to  the 
LMP  Program  Office,  these  claims  are  being  reviewed  by  the  Army  contracting  officer,  who 
is  expected  to  make  a  decision  by  June  2007. 
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software  development  to  a  commercial  ERP  solution,  as  well  as  a  change 
in  the  prime  contractor.  According  to  Army  officials,  it  invested 
approximately  $95  million  in  the  previous  efforts  before  adopting  the  ERP 
approach.  The  existing  ERP  effort  was  started  in  September  2003  and  is 
currently  estimated  to  reach  FOC  during  fiscal  year  2014.  Figure  2  shows 
the  time  line  for  GCSS-Army  as  of  March  2007. 


Figure  2:  GCSS-Army  Time  Line 
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Source:  DOD,  GCSS-Army  program  office. 


GCSS-Army  is  intended  to  replace  16  stovepiped,  legacy  logistics  systems 
that  cover  certain  types  of  inventory  and  PP&E.  The  system  is  intended  to 
be  operational  at  all  deployable  Army  units,  and  provide  asset  visibility  for 
accountable  items  down  to  the  foxhole.  As  of  September  30,  2006,  the 
Army  reported  that  it  had  obligated  approximately  $203  million  for  the 
ERP  version  of  GCSS-Army.  Additionally,  the  Army  estimates  that  another 
$2.1  billion  will  be  invested  in  GCSS-Army. 

Product  Lifecycle  Management  Plus  (PLM+).  This  initiative  is  the 
technical  enabler  to  integrate  LMP  and  GCSS-Army.  PLM+  is  the  means  by 
which  the  Army  intends  to  achieve  the  SALE  vision  of  integrated  logistics 
systems.  PLM+  is  intended  to  provide  a  single  point  of  entry  and  exit  for 
interfaces  to  external  systems.  Furthermore,  PLM+  is  intended  to 
eliminate  duplicative  and  costly  system  interfaces.  Additionally,  the  PLM+ 
implementation  schedule  and  related  funding  are  aligned  with  that  of 
GCSS-Army.  PLM+  does  not  stand  alone  as  an  independent  capability.  As 
of  September  2006,  the  Army  reported  that  approximately  $31  million  had 
been  obligated  for  PLM+. 
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GFEBS.  While  LMP  is  intended  to  provide  financial  control  for  the  Army 
Working  Capital  Fund,  GFEBS  is  intended  to  provide  this  important 
control  over  all  non- working  capital  fund  inventory,  including  that  which 
is  reported  in  GCSS-Army.  GFEBS  is  an  ERP  solution  that  was  initiated  in 
October  2004  and  is  intended  to  serve  as  the  Army’s  general  ledger  system 
for  its  general  fund  accounting.25  As  such,  GFEBS  is  intended  to  improve 
the  reliability  of  the  Army’s  financial  information  and  thereby  enhance  the 
Army’s  management  decision-making  process.  GFEBS  is  expected  to 
replace  87  legacy  systems,  including  the  30-year  old  Standard  Army 
Finance  System.  Figure  3  shows  the  GFEBS  time  line  as  of  June  2007. 


Figure  3:  GFEBS  Time  Line 
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Source:  DOD,  GFEBS  program  office. 


The  Army  estimates  that  GFEBS  will  reach  FOC  by  2010.  As  of 
September  30,  2006,  the  Army  reported  that  it  had  obligated  $123  million 
for  the  development  of  GFEBS.  In  addition,  the  Army  currently  estimates 
that  it  will  invest  another  $1.3  billion  to  implement  GFEBS. 


Overview  of  DOD’s  In  2006,  DOD  adopted  a  “tiered  accountability”  approach  to  improve 

Investment  Management  control  and  accountability  over  the  billions  of  dollars  it  invests  annually  in 
Practices  DOD  business  systems.  Under  this  approach,  executive  leadership  for  the 

direction,  oversight,  and  execution  of  DOD  investments  is  the 
responsibility  of  several  entities  within  DOD  and  its  components.  As 
shown  in  figure  4  and  described  below,  the  investment  control  process 
begins  at  the  component  level  and  works  its  way  up  through  a  hierarchy  of 


25The  Army  Corps  of  Engineers  will  continue  to  use  its  existing  financial  system — Corps  of 
Engineers  Financial  Management  System. 
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review  and  approval  authorities,  depending  on  the  size  and  significance  of 
the  investment.26 


26There  are  four  tiers  of  business  systems.  Tier  1  systems  include  all  large,  expensive 
system  programs  classified  as  a  “major  automated  information  system”  (MAIS)  or  a  “major 
defense  acquisition  program”  (MDAP)  and  subject  to  the  most  extensive  statutory  and 
regulatory  reporting  requirements.  Tier  2  systems  include  those  with  modernization  efforts 
of  $10  million  or  greater  but  that  are  not  designated  as  MAIS  or  MDAP  or  programs  that 
have  been  designated  as  IRB  interest  programs  because  of  their  effect  on  DOD 
transformation  objectives.  Tier  3  systems  include  those  with  modernization  efforts  that 
have  anticipated  costs  greater  than  $1  million  but  less  than  $10  million.  Tier  4  systems  are 
those  with  modernization  efforts  that  have  anticipated  costs  of  up  to  $1  million. 
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Figure  4:  Overview  of  Tiered  Accountability  for  Army  Business  System  Investments 


Source:  Army,  DOD. 


At  the  DOD  enterprise  level,  key  entities  involved  in  maintaining  control 
and  accountability  over  Army  business  system  investments  with  systems 
modernizations  over  $1  million  include  the  DBSMC,  which  serves  as  the 
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highest  ranking  governance  body  for  business  systems  modernization 
activities;  the  Principal  Staff  Assistants,  who  serve  as  the  certification 
authorities  for  business  system  modernizations  in  their  respective  core 
business  missions;  the  investment  review  boards  (IRBs),  which  form  the 
review  and  decision-making  bodies  for  business  system  investments  in 
their  respective  areas  of  responsibility  and  review  each  investment  for 
BEA  compliance;  and  BTA,  which  provides  support  to  the  DBSMC  and  the 
IRBs  and  is  responsible  for  leading  and  coordinating  business 
transformation  efforts  across  the  department.  The  BTA  is  organized  into 
seven  directorates,  one  of  which  is  the  Defense  Business  Systems 
Acquisition  Executive — the  component  acquisition  executive  for  DOD 
enterprise-level  (DOD-wide)  business  systems  and  initiatives.  This 
directorate  is  responsible  for  developing,  coordinating,  and  integrating 
DOD  enterprise-level  projects,  programs,  systems,  and  initiatives — 
including  managing  resources  such  as  funding,  personnel,  and  contracts 
for  assigned  systems  and  programs. 

To  implement  tiered  accountability  within  the  Army,  the  Army  designated 
its  Chief  Information  Officer  (CIO)  as  the  Army’s  Pre-Certification 
Authority  (PCA)  for  certification  and  annual  reviews  of  business  system 
investments.27  The  PCA  is  accountable  for  the  component’s  business 
system  investments  and  acts  as  the  component’s  principal  point  of  contact 
for  communication  with  the  IRBs.  As  such,  the  PCA  (1)  validates  that  the 
system  information  for  all  business  systems  modernizations  over 
$1  million  is  complete  and  accessible  to  the  IRBs,  (2)  reviews 
development/modernization  investments  with  total  cost  of  $1  million  or 
less,  (3)  reviews  system  compliance  with  DOD’s  BEA  and  enterprise 
transition  plan,  (4)  verifies  the  investment’s  economic  viability  analysis, 

(5)  asserts  the  status  and  validity  of  the  investment  information  by 
submitting  a  component  precertification  letter  to  the  appropriate  IRB  for 
its  review,  and  (6)  provides  IT  portfolio  management  policy  guidance  and 
oversight  of  mission  area/domain  IT  portfolios.  Below  the  Army  enterprise 
level,  the  Army  has  established  six  functional  area  domains  within  its 


^Certification  reviews  apply  to  business  system  development/modemization  programs 
with  a  total  cost  of  over  $1  million.  This  certification  review  focuses  on  program  alignment 
with  the  business  enterprise  architecture  and  must  be  completed  before  components 
obligate  funds  for  programs.  The  annual  review  applies  to  all  business  programs.  The  focus 
of  the  annual  review  is  to  determine  whether  the  system  development  effort  is  in 
compliance  with  the  business  enterprise  architecture,  meeting  its  milestones,  and 
addressing  IRB  certification  conditions. 
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business  mission  area.28  The  domains  are  responsible  for  implementing  the 
IT  portfolio  management  process  developed  by  DOD  and  the  Army  to 
define  and  justify  the  portfolio  of  planned  IT  expenditures  consistent  with 
strategic  objectives  and  operational  requirements. 


GAO  Has  Previously  Our  May  2004  report29  pointed  out  that  the  Army  had  not  effectively 

Reported  LMP  Issues  managed  its  implementation  of  LMP.  The  report  noted  that  after  LMP  was 

deployed  in  July  2003,  operational  difficulties  at  the  Tobyhanna  Army 
Depot  resulted  in  inaccurate  financial  management  information.  More 
specifically,  the  depot  was  not  (1)  producing  accurate  workload  planning 
information,  (2)  generating  accurate  customer  bills,  and  (3)  capturing  all 
repair  costs,  necessary  for  the  Army  to  calculate  accurate  future  repair 
prices.  The  report  also  pointed  out  that  LMP  requirements  (1)  lacked  the 
specific  information  necessary  to  understand  the  required  functionality 
that  was  to  be  provided,  and  (2)  did  not  describe  how  to  determine 
quantitatively,  through  testing  or  other  analysis,  whether  the  system  would 
meet  the  Army’s  needs.  Subsequently,  in  June  2005, 30  we  reported  that  the 
problems  with  LMP  continued  to  prevent  the  Tobyhanna  Army  Depot  from 
accurately  reporting  on  its  financial  operations,  including  gains  and  losses, 
which  adversely  affected  the  depot’s  ability  to  accurately  set  customer 
sales  prices.  Further,  the  report  pointed  out  that  problems  persisted  with 
recognizing  revenue  and  billing  customers.  We  recommended,  and  the 
Army  agreed,  that  the  implementation  of  LMP  should  be  delayed  until  the 
operational  problems  we  identified  were  resolved.  The  Army  is  continuing 
to  resolve  the  outstanding  issues. 


28DOD  defines  a  mission  area  as  a  defined  area  of  responsibility  with  functions  and 
processes  that  contribute  to  mission  accomplishment.  The  Ar  my  maps  its  mission  areas  to 
the  four  Global  Information  Grid  Enterprise  Services  Mission  Areas  delineated  by  DOD, 
including:  Warfighting,  Business,  Enterprise  Information  Environment,  and  the  DOD 
portion  of  National  Intelligence.  The  Business  Mission  Area  includes  six  domains: 

(1)  acquisition,  (2)  financial  management,  (3)  human  resources  management,  (4)  logistics, 
(5)  installations  and  environment,  and  (6)  civil  works. 

z9GAO-04-615. 

3°GAO-05-441. 
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Army  Lacks  an 
Integrated  Strategy 
for  Achieving  TAV 


The  Army  has  estimated  that  it  will  invest  about  $5  billion  over  the  next 
several  years  to  complete  development  and  implementation  of  GFEBS, 
GCSS-Army,  and  LMP.  However,  the  Army  is  making  this  significant 
investment  without  a  clear  integrated  strategy  for  how  these  systems  will 
be  used  to  achieve  TAV  over  hundreds  of  billions  of  dollars  of  assets.  As 
we  reported  in  December  2004, 31  because  DOD  has  not  developed  a  clear 
long-range  strategy,  the  military  services  will  be  exposed  to  the  risk  of 
spending  billions  of  dollars  on  duplicative,  stovepiped  systems  that  do  not 
support  the  department’s  business  transformation  goals,  including 
attaining  TAV.  More  specifically,  we  found  that  the  Army’s  current  strategy 
for  achieving  TAV,  including  the  implementation  of  these  three  systems, 
does  not  embrace  two  key  foundational  elements  that  we  had  identified  as 
essential  to  achieving  successful  transformation  of  business  systems  and 
processes:  an  EA  and  a  concept  of  operations.  Without  these  key 
foundational  elements,  the  Army  is  at  risk  of  investing  billions  of  dollars  in 
business  systems  that  may  not  achieve  DOD’s  and  the  Army’s  goal  of 
achieving  TAV.  Further,  the  Army’s  planned  strategy  perpetuates  some  of 
the  cumbersome  and  ineffective  business  processes  and  data 
redundancies  that  are  currently  being  used  in  the  existing  legacy  system 
environment. 


Army’s  Efforts  to  Achieve  The  Army  has  yet  to  develop  and  implement  an  Army  business  EA  to  help 
TAV  Lack  the  Benefit  Of  an  guide  its  system  efforts,  including  those  aimed  at  achieving  TAV.  DOD’s 
P  A  acquisition  policies  and  guidance,32  as  well  as  federal  and  best  practice 

guidance,33  recognize  the  importance  of  investing  in  business  systems 
within  the  context  of  an  EA.  GFEBS,  GCSS-Army,  and  LMP  are  not  being 
managed  and  developed  in  the  context  of  a  well-defined  Army-wide  EA.  In 


31GAO,  Defense  Inventory:  Improvements  Needed  in  DOD’s  Implementation  of  Its  Long- 
Term  Strategy  for  Total  Asset  Visibility  of  Its  Inventory ,  GAO-05-15  (Washington,  D.C.: 
Dec.  6,  2004). 

32DOD,  Department  of  Defense  Directive  Number  5000.1  (May  12,  2003);  Department  of 
Defense  Architecture  Framework,  Version  1.0,  Volume  1  (February  2004). 

33Pub.  L.  No.  104-106,  div.  E,  tit.  LI,  §§  5122,  5125;  E-Govemment  Act  of  2002,  Pub.  L.  No. 
107-347  §  101,  116  Stat.  2899,  2901-2910  (Dec.  17,  2002)  codified  at  44  U.S.C.  §§  3601-06); 
OMB  Circular  No.  A-130,  §  8(b);  GAO,  Information  Technology:  A  Framework  for 
Assessing  and  Improving  Enterprise  Architecture  Management  (Version  1.1), 
GAO-03-584G  (Washington,  D.C.:  April  2003);  Chief  Information  Officer  Council,  A 
Practical  Guide  to  Federal  Enterprise  Architecture,  Version  1.0  (February  2001);  and 
Institute  of  Electrical  and  Electronics  Engineers,  Standard  for  Recommended  Practice  for 
Architectural  Description  of  Software-Intensive  Systems  1471-2000  (Sept.  21,  2000). 
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August  2006, 34  we  reported  that  the  Army  was  in  the  very  early  stages  of 
developing  an  EA.  We  recommended  that  the  Secretary  of  Defense  ensure 
that  the  DOD  architecture  programs35  we  reviewed  develop  and  implement 
plans  for  fully  satisfying  each  of  the  conditions  in  our  Enterprise 
Architecture  Management  Maturity  Framework  (EAMMF)  for  assessing 
and  improving  EA  management.36  In  commenting  on  the  report,  DOD 
agreed  with  our  recommendation. 

As  detailed  in  our  August  2006  report,  we  assessed  the  Army’s  efforts  to 
develop  an  EA  against  the  criteria  specified  in  EAMMF.  Our  EAMMF  is  a 
five-stage  architecture  framework  for  managing  the  development, 
maintenance,  and  implementation  of  an  architecture  and  understanding 
the  extent  to  which  effective  architecture  management  practices  are  being 
performed  and  where  an  organization  is  in  its  progression  toward  having  a 
well-managed  architecture  program.  In  short,  the  framework  consists  of  31 
core  elements  that  relate  to  architecture  governance,  content,  use,  and 
measurement.  These  elements  reflect  research  by  us  and  others  showing 
that  architecture  programs  should  be  founded  upon  institutional 
architecture  commitment  and  capabilities,  and  measured  and  verified 
products  and  results.  Our  analysis  of  information  provided  by  the  Army 
indicated  that  it  had  satisfied  only  3  percent  of  all  framework  elements.  In 
essence,  this  means  that  the  Army  is  at  stage  1  of  developing  and 
implementing  an  EA.  While  stage  1  agencies  may  have  initiated  some 
enterprise  architecture  activity,  these  agencies’  efforts  are  ad  hoc  and 
unstructured,  lack  institutional  leadership  and  direction,  and  do  not 
provide  the  management  foundation  necessary  for  successful  enterprise 
architecture  development. 

In  May  2007,  we  reported  that  the  Army  still  had  not  developed  an  EA.37  As 
a  result,  the  Army  does  not  have  a  well-informed  basis  for  determining  if 
these  systems  will  fit  within  the  context  of  future  Army  business 
operations  and  will  most  efficiently  and  effectively  address  the  Army’s 


34GAO-06-831. 

3BThe  five  DOD  architecture  programs  that  were  included  in  our  audit  were  the 
Departments  of  the  Army,  Navy,  and  Air  Force,  and  the  DOD  business  enterprise 
architecture  and  the  DOD  global  information  grid. 

36GAO,  Information  Technology:  A  Framework  for  Assessing  and  Improving  Enterprise 
Architecture  Management  (Version  1.1),  GAO-03-584G  (Washington,  D.C.:  April  2003). 

37GAO-07-733. 
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long-standing  weaknesses  associated  with  the  lack  of  asset  visibility. 
Improving  asset  visibility  is  critical  to  addressing  the  problems  associated 
with  supply  chain  management,  which  has  been  on  our  high-risk  list  since 
1990.38  Asset  visibility  is  one  of  the  focus  areas  that  are  a  critical  part  of 
supply  chain  management.39 

In  addition,  without  a  clear  understanding  of  its  current  systems  and 
business  processes — commonly  referred  to  as  the  “As  Is”  environment — 
and  the  business  operations  it  envisions  for  the  future — commonly 
referred  to  as  the  “To  Be”  environment — the  Army  will  increase  the  risk 
that  misalignments  can  occur  that  can  introduce  redundancies  and 
incompatibilities  that  can  produce  inefficiencies  and  require  costly  and 
time-consuming  rework  to  fix.  GFEBS,  GCSS-Army,  and  LMP  have 
proceeded  without  a  common,  institutional  frame  of  reference  (for 
example,  EA)  that  can  be  used  to  effectively  manage  their  relationships 
and  dependencies.  In  November  2006,  DOD’s  internal  review  of  GFEBS  by 
BTA  noted  that  the  Army  lacked  a  strategy  for  integrating  GFEBS,  GCSS- 
Army,  and  LMP. 


Army  Strategy  to  Achieve 
TAV  Does  Not  Utilize  an 
Enterprisewide 
Perspective 


The  Army  also  lacks  a  concept  of  operations  for  how  GFEBS,  GCSS-Army, 
and  LMP  will  collectively  achieve  TAV.  An  effective  concept  of  operations 
would  describe,  at  a  high  level,  (1)  how  the  three  business  systems  relate 
to  each  other  in  achieving  the  Army’s  TAV  goal,  and  (2)  how  information 
flows  from  and  through  these  systems.  Moreover,  we  found  that  the 
Army’s  lack  of  a  concept  of  operations  has  contributed  to  its  failure  to 
take  full  advantage  of  business  process  reengineering  opportunities  that 
are  available  when  using  an  ERP  solution.  Without  a  clear  long-term 
strategy,  the  Army  lacks  a  key  management  control  for  ensuring  that  time 
frames,  results-oriented  performance  measures,  and  accountability 
mechanisms  are  established  and  monitored  to  help  achieve  TAV. 


Specifically,  we  noted  that  the  Army’s  strategy  perpetuates  some  of  the 
cumbersome  and  inefficient  business  processes  that  are  currently  being 
used  in  the  existing  legacy  system  environment.  One  of  the  key  benefits  of 
an  ERP  is  that  it  can  be  used  to  streamline  business  processes  and 
eliminate  data  redundancy.  However,  some  of  the  basic  business 


38GAO,  High-Risk  Series:  An  Update,  GAO-07-310  (Washington,  D.C.:  January  2007). 

39The  other  two  focus  areas  related  to  supply  chain  management  are  requirements 
forecasting  and  materiel  distribution. 
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processes  being  used  or  expected  to  be  used  by  GFEBS,  GCSS-Army,  and 
LMP  are  based  on  the  Army’s  existing  business  processes — many  of  which 
are  error  prone,  labor  intensive,  and  redundant.  Considering  the  billions  of 
dollars  the  Army  is  investing  in  these  systems,  it  is  incumbent  upon  the 
Army  to  embrace  and  utilize  the  most  efficient  and  streamlined  ERP 
processes  to  the  fullest  extent  possible  in  its  effort  to  achieve  TAV. 

For  example,  the  Army’s  planned  strategy  to  improve  control  and 
accountability  over  its  PP&E  does  not  take  advantage  of  the  capabilities  of 
an  ERP  solution.  (See  app.  Ill  for  additional  examples  in  which  the  Army’s 
TAV  systems  initiatives  have  not  effectively  streamlined  legacy  processes 
related  to  funds  control  and  disbursements.)  Financial  data,  including 
acquisition  cost  and  depreciation,  for  most  of  the  Army’s  PP&E  are 
currently  maintained  in  the  Defense  Property  Accountability  System 
(DPAS).  PP&E  accountability  data,  such  as  location  and  quantity,  are 
maintained  in  a  variety  of  legacy  systems  such  as  the  Property  Book  Unit 
Supply  Enhanced  (PBUSE)  system — which  is  1  of  the  16  systems  that  is 
expected  to  be  replaced  by  GCSS-Army.  In  March  2007,  Army  Program 
Executive  Office  Enterprise  Information  Systems  officials  informed  us  that 
the  financial  data — such  as  acquisition  cost  and  depreciation — for  the 
Army’s  PP&E  will  be  transferred  from  DPAS  to  GFEBS.  This  maintains  the 
existing  accounting  data  in  an  accounting  system.  On  the  other  hand,  the 
accountability  data — such  as  location,  condition,  and  number  of  units — for 
PP&E  assets  will  initially  be  transferred  from  the  current  legacy  system  to 
PBUSE  or  GCSS-Army,  depending  on  whether  the  items  belong  to  an 
installation  or  an  Army  unit  that  may  be  deployed.  The  transferring  of 
items  to  PBUSE  appears  to  be  an  interim  solution  since,  according  to 
DOD’s  enterprise  transition  plan  GCSS-Army  is  to  replace  PBUSE. 
However,  GCSS-Army  is  not  expected  to  assume  the  control  and 
accountability  over  items  that  are  recorded  in  PBUSE  associated  with  the 
Army  Working  Capital  Fund  or  other  Army  organizational  entities  that  are 
not  considered  to  be  units  that  can  be  deployed  (field/tactical  units)  in 
military  operations. 

The  Army’s  decision  to  segregate  its  PP&E  financial  and  accountability 
data  into  two  separate  system  solutions  reflects  its  lack  of  an  enterprise 
concept  of  operations  and  enterprise  view  for  achieving  TAV.  The  Army’s 
strategy  of  moving  PP&E  data  from  DPAS  to  PBUSE,  and  ultimately  to 
another  system,  means  that  one  data  conversion  will  take  place  to  move 
the  data  from  DPAS  to  PBUSE — a  system  the  Army  is  supposed  to 
eliminate  with  the  implementation  of  GCSS-Army — followed  by  another 
data  conversion  from  PBUSE  to  yet  another  system,  not  yet  identified. 
Further,  the  Army  is  continuing  to  limit  the  capabilities  of  LMP  by 
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transferring  all  information  on  PP&E  from  DPAS  to  PBUSE — which  as 
noted  before  is  considered  a  legacy  business  system.  As  a  result, 
information  on  PP&E  that  could  enhance  Tobyhanna’s  operations, 
especially  in  the  area  of  workload  planning,  will  not  be  available  within  the 
LMP  environment. 

Thus,  the  approach  being  taken  by  the  Army  is  simply  to  adapt  the 
processes  used  by  the  legacy  systems  rather  than  use  the  streamlined 
processes  inherent  in  the  ERP  solution.  If  the  functionality  of  an  ERP 
solution  was  fully  utilized  by  the  Army,  it  would  increase  the  likelihood 
that  TAV  could  be  accomplished  within  each  of  the  three  systems  for  the 
specific  type  of  PP&E  items  each  was  responsible  for.  For  example, 

GFEBS  would  contain  both  the  financial  and  accountability  information 
related  to  nontactical/non-working  capital  fund  PP&E,  GCSS-Army  would 
have  all  the  information  associated  with  tactical  PP&E,  and  LMP  would 
have  the  data  related  to  Working  Capital  Fund  PP&E.  Figure  5  illustrates 
one  way  that  an  ERP  approach  could  be  utilized  to  achieve  physical  and 
financial  accountability  control  of  the  Army’s  PP&E. 


Figure  5:  Example  of  an  ERP  Vision  for  Accountability  of  the  Army’s  PP&E 
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Source:  GAO. 


While  the  Army’s  approach  may  result  in  incremental  improvements  in  its 
asset  accountability,  it  does  not  take  full  advantage  of  the  (1)  benefits  of 
adopting  enterprise  processes  and  (2)  functionality  that  is  available  in  the 
ERP  solution  it  has  selected  to  support  those  processes.  Fundamental  to  a 
successful  ERP  implementation  is  the  reengineering  of  an  organization’s 
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business  processes  in  a  manner  that  helps  ensure  that  the  right  resources 
(people,  material,  machinery,  and  funds)  are  available  in  the  correct 
quantities  when  needed.40  While  Army  has  stated  that  its  use  of  ERP 
software  would  help  reengineer  its  business  processes,  we  found  that  at 
least  some  business  processes  that  are  being  used  or  expected  to  be  used 
in  the  future  do  not  reflect  reengineered  processes  necessary  to  most 
effectively  implement  an  ERP  solution.  By  perpetuating  the  use  of 
cumbersome,  error-prone,  and  ineffective  business  processes  in  its  asset 
accountability  operations,  the  Army  will  diminish  its  capability  to  achieve 
TAV  and  improve  accountability  over  its  assets. 


Army’s  Ability  to 
Effectively  Oversee 
Portfolios  of  Business 
Systems  Investment  Is 
Not  Yet  Fully 
Developed 


While  the  Army  has  established  a  governance  structure  to  oversee  its 
business  system  investments — including  its  asset  accountability  system 
investments — that  is  consistent  with  DOD  guidance,  additional  actions  are 
needed  to  enhance  oversight,  control,  and  accountability.  The  Army’s 
business  system  investment  oversight  efforts  to  date  have  primarily 
focused  on  ensuring  that  business  systems  modernizations  over  $1  million 
are  reviewed  in  accordance  with  the  criteria  specified  in  the  fiscal  year 
2005  National  Defense  Authorization  Act.  Both  DOD  and  Army  officials 
acknowledged  that  the  department’s  and  Army’s  investment  review 
processes,  particularly  related  to  their  ability  to  review  business  system 
investments  from  a  portfolio  perspective,  are  in  the  early  stage  of  maturity. 
Portfolio  management  is  a  conscious,  continuous,  and  proactive  approach 
to  allocating  limited  resources  among  competing  initiatives  in  light  of  the 
investments’  relative  benefits.  Taking  an  enterprise  view  enables  an 
organization  to  consider  its  investment  comprehensively,  so  that 
collectively  the  investments  optimally  address  the  organization’s  mission, 
goals,  and  objectives.  In  addition,  we  found  that  the  Army  did  not  have 
reliable  processes  and  analyses  to  support  its  oversight  of  individual 
systems  modernization  program  efforts  intended  to  improve  asset 
visibility.  Specifically,  (1)  the  Army  has  not  established  or  implemented  an 
independent  verification  and  validation  (IV&V)  process  for  the  three 
systems — GFEBS,  GCSS-Army,  or  LMP  and  (2)  the  August  2004  GFEBS 


40Thomas  F.  Wallace  and  Michael  H.  Kremzar,  ERP:  Making  It  Happen,  (New  York,  N.Y.: 
John  Wiley  &  Sons,  Inc.  2001). 


Page  22 


GAO-07-860  Army  Asset  Visibility 


economic  analysis  was  not  prepared  in  accordance  with  DOD  and  OMB 
guidance.41 


Army’s  Oversight  of  Its 
Business  System 
Investments  Lacks  a 
Portfolio  Perspective 


The  Army’s  oversight  of  its  business  system  investments — including 
system  investments  intended  to  achieve  TAV — continues  to  be  focused  on 
the  review  and  approval  of  individual  business  system  investments.  In  May 
2007,  we  reported  that  DOD  needed  to  improve  its  policies  and  procedures 
associated  with  managing  its  business  system  investments  as  portfolios.42 
More  specifically,  we  found  that  DOD  had  not  yet  progressed  from  project- 
based  processes  to  portfolio-based  processes,  a  key  element  for  effectively 
managing  business  system  transformation  efforts.  DOD  informed  us  that 
DOD  components  are  responsible  for  developing  and  managing  their  own 
portfolio  management  processes. 

Army  IT  guidance  for  managing  IT  investments  requires  portfolios  to  be 
managed  and  monitored  using  established  quantifiable  outcome-based 
performance  measures  to  determine  whether  to  recommend 
continuations,  modification,  or  termination  of  individual  investments 
within  a  portfolio.43  However,  the  Army  has  not  implemented  processes  or 
procedures  that  facilitate  an  enterprise  view  toward  management  and 
oversight  of  portfolios  of  business  system  investments  intended  to 
collectively  provide  a  specific  capability  or  functionality.  Rather,  the 
Army’s  current  business  system  investment  process  focuses  primarily  on 
reviewing  and  approving  investment  packages  submitted  by  the 
responsible  program  management  office  for  individual  systems 
improvement  efforts  that  have  development/modernization  funding  over 
$1  million.  Individual  business  system  investments  that  exceed  the 


J1An  economic  analysis  of  Army’s  LMP  initiative  is  not  required  because  it  is  considered  a 
procurement  of  a  service  as  opposed  to  an  actual  system,  and  therefore  is  not  subject  to 
the  criteria  established  by  Department  of  Defense  Directive  5000.1  and  Department  of 
Defense  Instruction  5000.2.  Subsequent  to  the  draft  of  this  report  being  submitted  to  DOD 
for  comment,  we  were  infonned  that  the  Office  of  the  Deputy  Assistant  Secretary  of  the 
Army  for  Cost  and  Economics  had  determined  that  GCSS-Army  is  economically  viable.  We 
have  not  evaluated  the  results  of  that  analysis. 

42GAO,  Business  Systems  Modernization:  DOD  Needs  to  Fully  Define  Policies  and 
Procedures  for  Institutionally  Managing  Investments,  GAO-07-538  (Washington,  D.C.: 
May  11,  2007). 

43U.S.  Army,  Army  Knowledge  Management  (AKM)  Guidance  Memorandum,  Capabilities- 
Based  Information  Technology  (IT)  Portfolio  Governance  Implementing  Guidance 
(Jan.  5,  2006). 
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$1  million  systems  modernization  threshold  are  required  to  be  reviewed  by 
the  appropriate  IRB  and  approved  by  the  DBSMC. 

Army  officials  acknowledged  that  the  Army’s  ability  to  provide  portfolio- 
based  investment  review  and  oversight  is  in  the  early  stages  of 
development.  For  example,  while  the  Army  has  begun  a  process  to  share 
investment  package  information,  including  cost,  schedule,  and 
performance  data,  across  its  mission  areas  and  domains,  Army  officials 
acknowledged  that  the  information  provided  by  a  program  management 
office,  as  well  as  the  associated  oversight  and  review,  is  centered  largely 
on  the  functionality  and  capabilities  of  individual  systems.  For  example, 
annual  certifications  for  the  three  systems,  LMP,  GCSS-Army,  and  GFEBS, 
which  the  Army  intends  to  utilize  collectively  in  achieving  TAV,  are  not 
submitted  or  reviewed  as  a  portfolio,  but  rather  as  individual  system 
investments.  The  Army  has  not  yet  implemented  the  processes  to  evaluate 
and  improve  its  progress  toward  achieving  TAV  using  portfolio-based 
projects  such  as  costs,  schedule,  performance,  and  risks.  As  a  result,  the 
Army’s  ability  to  effectively  oversee  the  development  of  the  portfolio  of 
systems  intended  to  collectively  provide  the  Army  with  TAV  is  limited. 
Until  the  Army’s  governance  process,  policies,  and  procedures  mature, 
including  its  ability  to  apply  a  corporate  portfolio  perspective  in  managing 
and  overseeing  subportfolios  of  business  system  investments  intended  to 
provide  a  desired  capability  or  function,  the  Army  is  at  risk  of 
implementing  systems  that  (1)  do  not  provide  the  desired  capability, 
including  TAV;  (2)  are  stovepiped  in  their  functionality;  (3)  do  not 
interoperate  in  an  efficient  manner;  and  (4)  are  not  supported  by  efficient 
and  effective  business  processes. 


Reliable  Processes  and 
Analyses  Are  Needed  to 
Facilitate  Management 
Oversight  of  Business 
System  Investments 


The  Army  has  not  implemented  reliable  processes  and  analyses  needed  to 
enhance  its  management  oversight  of  program  management  office  efforts 
to  develop  and  implement  its  TAV,  as  well  as  other  business  systems  on 
time,  within  budget,  and  with  the  intended  capability.  Both  DOD’s  and  the 
Army’s  business  system  investment  oversight  processes  are  highly 
dependent  upon  the  reliability  of  investment  information  provided  by 
Army  program  management  offices.  Thus,  the  reliability  of  that  business 
system  investment  information  is  critical  to  both  DOD’s  and  the  Army’s 
ability  to  effectively  oversee,  manage,  and  redirect — if  necessary — the 
services’  business  system  development  and  modernization  efforts. 
However,  we  found  that  the  Army  had  not  yet  established  an  independent 
verification  and  validation  function  for  any  of  the  three  systems  we 
reviewed.  In  addition,  the  economic  analysis  submitted  by  the  program 
management  office  for  GFEBS  was  not  prepared  in  accordance  with  DOD 
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Independent  Verification  and 
Validation  Would  Enhance 
Program  Oversight 


or  OMB  guidance.  As  a  result,  the  economic  analysis  submitted  by  the 
program  management  office  to  DOD  and  the  Army  oversight  entities 
justifying  the  Army’s  investment  in  GFEBS  was  based  on  questionable 
costs  and  benefits. 

The  Army  had  not  yet  established  an  IV&V  function44  for  any  of  the  three 
systems — GFEBS,  GCSS-Army,  and  LMP.  While  the  Army  has  established  a 
verification  and  validation  function  for  LMP,  it  was  not  an  independent 
review  because  the  reviewer  reports  directly  to  the  LMP  program 
management  office.  Independence  is  a  key  component  to  reliable 
verification  and  validation  function. 

Best  business  practices  have  demonstrated  that  use  of  an  IV&V  function  is 
an  invaluable  means  to  providing  management  reasonable  assurance  that  a 
planned  system,  or  the  portfolio  of  systems,  will  satisfy  its  planned  use  and 
users.  An  effective  IV&V  review  process  should  provide  an  independent 
assessment  to  DOD  and  Army  management  of  the  overall  status  of  the 
project,  including  a  discussion  of  any  existing  or  potential  revisions  to  the 
project  with  respect  to  cost,  schedule,  and  performance.  The  IV&V  reports 
should  identify  to  management  the  issues  or  weaknesses  that  increase  the 
risks  associated  with  the  project  or  portfolio  to  senior  management  so  that 
they  can  be  promptly  addressed.  These  assessments  involve  reviewing 
project  documentation,  participating  in  meetings  at  ah  levels  within  the 
project,  and  providing  periodic  reports  and  recommendations,  if  deemed 
warranted,  to  senior  management.  The  IV&V  function  should  report  on 
every  facet  of  a  system  project  such  as: 

Testing  program  adequacy.  Testing  activities  would  be  evaluated  to 
ensure  they  are  properly  defined  and  developed  in  accordance  with 
industry  standards  and  best  practices. 

Critical-path  analysis.  A  critical  path  defines  the  series  of  tasks  that 
must  be  finished  in  time  for  the  entire  project  (or  portfolio  of  projects)  to 
finish  on  schedule.  Each  task  on  the  critical  path  is  a  critical  task.  A 


44According  to  the  Institute  of  Electrical  and  Electronics  Engineers,  verification  and 
validation  processes  for  projects  such  as  the  Army  ERPs  can  be  used  to  determine  whether 
(1)  the  products  of  a  given  activity  conform  to  the  requirements  of  that  activity  and  (2)  the 
software  satisfies  its  intended  use  and  user  needs.  This  determination  may  include 
analyzing,  evaluating,  reviewing,  inspecting,  assessing,  and  testing  software  products  and 
processes.  The  verification  and  validation  processes  should  assess  the  software  in  the 
context  of  the  system,  including  the  operational  environment,  hardware,  interfacing 
software,  operators,  and  users. 
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Validity  of  GFEBS’s  Economic 
Analysis  Questionable 


critical-path  analysis  helps  to  identify  the  impact  of  various  project  events, 
such  as  delays  in  project  deliverables,  and  ensures  that  the  impact  of  such 
delays  is  clearly  understood  by  all  parties  involved  with  the  project(s). 

System  strategy  documents.  Numerous  system  strategy  documents  that 
provide  the  foundation  for  the  system  development  and  operations  are 
critical  aspects  of  an  effective  system  project.  These  documents  are  used 
for  guidance  in  developing  documents  for  articulating  the  plans  and 
procedures  used  to  implement  a  system.  Examples  of  such  documents 
include  the  Life-cycle  Test  Strategy,  Interface  Strategy,  and  Conversion 
Strategy. 

Our  analysis  of  the  August  2004  GFEBS  economic  analysis  found  that  it 
was  not  prepared  in  accordance  with  OMB  and  DOD  guidance.45  The 
purpose  of  an  economic  analysis  is  to  give  the  decision  maker  insight  into 
economic  factors  that  have  a  bearing  on  accomplishing  the  stated 
objective  of  the  system  investment,  for  example  implementation  of  GFEBS 
to  improve  the  Army’s  financial  accountability.  As  such,  it  is  important 
that  the  economic  analysis  reliably  identifies  factors,  such  as  cost  and 
performance  risks  and  drivers  that  can  be  used  to  establish  priorities  and 
allocate  resource  allocations.  While  it  may  be  appropriate  to  invest  in  a 
particular  IT  investment,  such  as  GFEBS,  for  reasons  other  than  estimated 
economic  benefits,  nonetheless,  the  issues  we  identified  raise  questions  as 
whether  the  funds  invested  in  GFEBS  were  economically  justified.  Our 
specific  concerns  with  the  August  2004  GFEBS  economic  analysis  are 
highlighted  below. 

Inappropriate  cost  savings.  At  least  $142  million  of  estimated  savings 
were  not  savings,  but  rather,  should  have  been  classified  as  transfers — 
which  do  not  equate  to  economic  benefits.  Transfers  represent  shifts  of 
control  over  resource  allocation  from  one  group  to  another  that  do  not 
result  in  a  net  change  in  the  value  of  the  resources  involved  in  the  transfer. 
For  example,  we  found  that  the  Army  claimed  over  $88  million  of  savings 
related  to  transferring  real  property  management  from  the  Army’s  legacy 
system  to  GFEBS.  Since  the  Army’s  real  property  management 
responsibilities  were  not  eliminated,  only  transferred  from  the  legacy 
system  environment  to  GFEBS,  this  represents  a  transfer  rather  than  an 


45OMB  Circular  No.  A-94,  Guidelines  and  Discount  Rates  for  Benefit-Cost  Analysis  of 
Federal  Programs  (Oct.  29,  1992);  Department  of  Defense  Instruction  7041.3,  Economic 
Analysis  for  Decisionmaking  (Nov.  7,  1995);  Department  of  the  Army,  U.S.  Army  Cost  and 
Economic  Analysis  Center,  Economic  Analysis  Manual  (February  2001). 
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economic  benefit  for  the  Army.  Additionally,  the  Army  claimed  benefits  of 
about  $54  million  related  to  GFEBS  reducing  the  length  of  time  funds  are 
borrowed  by  the  Treasury  to  meet  the  Army  financial  obligations  by 
facilitating  the  Army’s  ability  to  make  just-in-time  disbursements.  OMB 
guidance  provides  that  there  are  no  economic  gains  from  a  pure  transfer 
payment  because  the  benefits  to  those  government  entities  that  receive 
such  a  transfer  are  matched  by  the  costs  borne  by  those  government 
entities  that  provide  the  transfer.46  The  Army’s  inappropriate  classification 
of  transfers  resulted  in  an  overstatement  of  the  economic  benefits  that 
would  be  achieved  by  implementing  GFEBS.  Additionally,  in  November 
2006,  BTA’s  enterprise  risk  assessment  management  review  of  GFEBS 
questioned  justifying  the  GFEBS  program  on  factors  not  fully  within  the 
program’s  control. 

•  Lack  of  sensitivity  analysis.  The  analysis  did  not  include  an  assessment 
of  the  effects  of  the  uncertainty  inherent  in  estimates  of  GFEBS  benefits 
and  costs,  as  stipulated  in  OMB  and  DOD  guidance.47  Because  an  economic 
analysis  uses  estimates  and  assumptions,  it  is  critical  that  a  sensitivity 
analysis  be  performed  to  understand  the  effects  of  the  imprecision  in  both 
underlying  data  and  modeling  assumptions.  This  analysis  is  required  since 
the  estimates  of  future  benefits  and  costs  are  subject  to  varying  degrees  of 
uncertainty.  Sensitivity  analysis  refers  to  changing  the  value  of  a  given 
variable  in  a  model  to  gauge  the  effect  of  change  on  model  results.  It  varies 
a  single  data  element  or  assumption  while  holding  the  other  data  elements 
and  assumptions  constant  to  determine  what  amount  of  change  in  that 
element  is  required  to  raise  or  lower  the  resulting  benefit  and  cost 
elements.  In  this  way,  GFEBS  data  and  assumptions  can  be  risk-ranked  for 
decisionmaking  and  auditing.  The  Army’s  failure  to  conduct  a  sensitivity 
analysis  of  GFEBS,  as  part  of  its  economic  analysis,  to  identify  the  effect 
of  uncertainties  associated  with  different  assumptions  increases  the 
chance  that  decisions  regarding  GFEBS  will  be  made  without  a  clear 
understanding  of  the  possible  impact  on  GFEBS  estimates  of  costs  and 
benefits.  Army  officials  informed  us  that  they  will  consider  including  a 
sensitivity  analysis  of  the  effect  of  uncertainties  when  they  update 
GFEBS’s  economic  analysis. 

•  Lack  of  documentation  supporting  benefits  and  cost  assumptions. 

The  GFEBS  program  management  office  could  not  provide  us  with 
benefits  and  costs  data  needed  to  replicate  their  analyses.  As  a  result,  we 


46OMB  Circular  No.  A-94,  §  6(a)(4). 

47OMB  Circular  No.  A-94,  §  9;  Department  of  Defense  Instruction  7041.3,  §  E3.2.2. 
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were  unable  to  trace  the  cost  and  benefit  estimates  reported  in  the  GFEBS 
economic  analysis  to  original  source  documentation  to  assess  and  validate 
the  reliability  and  applicability  of  the  data.  OMB  and  DOD  guidance  stress 
the  importance  of  maintaining  documentation  supporting  assumptions 
used  in  a  cost-benefit  analysis. 


LMP  Illustrates 
Continuing  Problems 
in  Implementing 
Business  Systems  on 
Time,  within  Budget, 
and  with  the 
Promised  Capability 


LMP  continues  to  be  plagued  by  operational  problems  that  have  beset  the 
system  virtually  since  its  initial  implementation  in  July  2003.  While,  an  EA, 
concept  of  operations,  and  effective  IT  portfolio  management  are  essential 
elements  in  the  Army’s  efforts  to  transform  its  operations,  it  is  equally 
important  for  the  Army  to  have  disciplined  processes  in  place  to 
implement  its  business  systems  on  time,  within  budget,  and  with  the 
promised  capability.  Many  of  DOD’s  approximately  2,980  business  systems 
are  nonintegrated,  stovepiped,  and  not  capable  of  providing  departmental 
management  and  the  Congress  accurate  and  reliable  information  on  DOD’s 
day-to-day  operations.  LMP  is  no  exception.  As  previously  noted,  LMP  was 
to  have  reached  FOC  in  fiscal  year  2005,  but  currently  FOC  is  estimated  for 
fiscal  year  2010 — a  slippage  of  5  years.  Further,  LMP’s  estimated  program 
costs  have  also  increased.  Tobyhanna  Army  Depot — the  only  depot  that  is 
operating  LMP — and  DFAS  personnel  have  stated  that  improvements  have 
been  made  in  LMP’s  operating  efficiency.  However,  problems  in  the  areas 
of  revenue  recognition  and  billing  continue.  These  problems  can,  in  part, 
be  attributed  to  ineffective  system  testing.  The  operational  issues 
confronting  LMP  and  our  concerns  about  the  effectiveness  of  the  system 
testing  are  highlighted  below  and  discussed  in  more  detail  in  appendix  IV. 


Operational  Issues  Tobyhanna  Army  Depot  continues  to  experience  problems  with  LMP 

Continue  accurately  recognizing  revenue  and  billing  customers.  These  problems 

have  existed  virtually  since  the  implementation  of  LMP  in  July  2003.  While 
the  Army  and  its  contractor — Computer  Sciences  Corporation  (CSC) — 
have  made  numerous  attempts  to  fix  the  problem,  they  have  not  been 
successful.  Additionally,  the  accuracy  of  LMP  financial  reports  continues 
to  be  questionable.  For  instance,  the  DOD  Inspector  General’s  audit  of  the 
Army’s  Working  Capital  Fund  financial  statements  for  fiscal  year  2006 
noted  that  LMP  was  not  properly  recording  transactions  in  accordance 
with  the  U.S.  Government  Standard  General  Ledger  requirements. 

For  the  3-month  period  ending  January  31,  2007,  we  found  that  LMP’s 
continuing  billing  problems  resulted  in  (1)  customers  not  being  billed  for 
costs  incurred  that  should  have  been  billed  and  (2)  customers  being  billed 
too  much.  Based  upon  the  billing  information  provided  by  the  LMP 
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program  management  office,  there  were  146  customer  orders  valued  at 
approximately  $6.4  million  that  were  not  billed  (or  recognized  as  revenue) 
during  the  January  31,  2007,  billing  cycle.  Customer  billings  and  the 
associated  revenue  provide  the  means  by  which  the  depot  finances  its  day- 
to-day  operations.  Similarly,  during  the  January  31,  2007,  billing  cycle 
about  308  customer  orders,  amounting  to  about  $5.8  million,  were  shown 
as  being  overbilled  and  a  credit  should  have  been  provided  to  the 
customer.  Moreover,  the  continuing  billing  and  revenue  recognition 
problem  may  be  a  factor  contributing  to  higher  depot  bills  from  DFAS. 
According  to  DFAS  personnel,  the  recurring  billing  problems  have  resulted 
in  DFAS  personnel  processing  Tobyhanna  Army  Depot’s  bills  manually. 
Based  on  information  provided  by  DFAS,  the  number  of  hours  spent  to 
provide  DFAS  accounting  services — including  billings  for  the  Tobyhanna 
Army  Depot — has  increased  from  approximately  17,800  hours  in  fiscal 
year  2004  to  over  22,600  hours  in  fiscal  year  2006. 


System  Testing  Remains  a 
Concern 


Our  analysis  of  10  selected  LMP  corrective  actions  identified  specific 
testing  weaknesses  in  each  of  the  corrective  actions  reviewed.  For 
example,  none  of  the  test  scripts  provided  adequate  information  that 
linked  the  items  tested  to  the  specific  requirement  being  tested.  This 
linking  is  commonly  referred  to  as  traceability  and  is  characteristic  of  a 
disciplined  testing  process.  The  test  scripts  reviewed  contained  headings 
for  information  on  the  scenario,  key  data  requirements,  expected  results, 
actual  results,  and  whether  the  test  was  considered  successful.  However, 
the  actual  test  script  did  not  contain  the  level  of  specificity  that  clearly 
delineated  how  a  specific  requirement  identified  in  the  requirement 
document(s)  and  the  associated  requirement(s)  in  that  document  were 
being  tested.  As  a  result,  it  was  not  possible  to  determine  if  all  LMP  system 
requirements  were  properly  tested.  Without  linking  a  given  requirement  to 
the  tests  designed  to  exercise  that  requirement,  it  was  impossible  to  obtain 


Our  prior  audits48  of  LMP  identified  significant  weaknesses  with  the  LMP 
program  management  office’s  efforts  to  effectively  implement  the 
requirements  management  and  testing  processes  needed  to  reduce  risks  to 
acceptable  levels.  During  our  current  audit,  we  found  that  the 
requirements  management  processes  have  improved — a  critical  first 
step — but  concern  remains  with  respect  to  the  adequacy  of  the  testing. 


48GAO-04-615  and  GAO-05-441. 
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reasonable  assurance  that  (1)  all  the  requirements  were  tested  and  (2)  the 
applied  test  provided  adequate  coverage  for  each  requirement. 

After  discussing  our  concerns  with  the  LMP  program  management  office 
and  its  contractor  in  February  2007,  the  LMP  program  office  requested  that 
its  verification  and  validation  (V&V)  contractor  perform  an  assessment  of 
the  LMP  testing  process.  According  to  information  provided  by  the  LMP 
program  office  and  the  V&V  contractor,  in  March  2007,  the  review  found 
that  the  LMP  testing  process  was  adequately  planned  but  that  it  did  not 
support  a  clear  understanding  of  the  (1)  requirement  being  tested  and 
(2)  tests  used  to  determine  whether  a  requirement  was  adequately 
implemented.  The  V&V  contractor’s  assessment  substantiates  our  analysis 
of  the  LMP  testing  process.  According  to  the  V&V  contractor,  these  areas 
need  to  be  addressed  before  the  Army  Test  and  Evaluation  Command 
conducts  its  review  of  LMP. 

According  to  LMP  program  management  office  officials,  the  CSC  is 
developing  and  implementing  a  testing  process  that  is  designed  to  address 
the  concerns  raised  by  us  and  the  Army  V&V  contractor.  CSC  is  expected 
to  have  a  testing  program  that  clearly  links  the  requirements  to  the  tests 
being  conducted  and  to  establish  an  independent  test  group  to  ensure  that 
the  testing  process  is  following  best  practices.  However,  until  the  Army 
has  reasonable  assurance  that  an  effective  testing  process  has  been 
properly  implemented,  we  are  reiterating  our  June  2005  recommendation 
that  LMP  not  be  deployed  to  additional  locations  until  the  Army  has 
assurance  that  LMP  is  providing  the  intended  functionality  and,  more 
specifically,  that  LMP  can  accurately  bill  its  customers  and  recognize 
revenue. 


Conclusions 


The  Army’s  efforts  to  develop  and  implement  GFEBS,  GCSS-Army,  and 
LMP  may  result  in  incremental  improvements  in  the  Army’s  accountability 
and  visibility  over  its  billions  of  dollars  in  tangible  assets.  However,  the 
Army  has  not  developed  or  utilized  key  management  tools  and  concepts 
necessary  to  successfully  transform  its  business  processes  to  achieve  TAV. 
The  primary  question  that  the  Army  has  yet  to  effectively  consider  and 
address  is  how  these  systems  and  associated  processes,  individually  and 
collectively,  will  provide  the  desired  functionality  necessary  to  achieve 
TAV.  Until  the  Army  develops  and  implements  an  Army  EA,  concept  of 
operations,  and  portfolio-based  management  and  oversight  processes,  it 
continues  to  risk  (1)  investing  billions  of  dollars  in  asset  accountability 
systems  that  may  not  enhance  the  department’s  and  the  Army’s  goal  of 
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achieving  TAV,  and  (2)  further  delay  DOD’s  efforts  to  remove  supply  chain 
management  from  our  high-risk  list. 

It  is  also  important  that  the  Army  take  action  to  implement  the  disciplined 
processes  necessary  to  implement  asset  accountability  systems  on  time, 
within  budget,  and  with  the  promised  capability.  Continuing  problems 
with  LMP  are  illustrative  of  the  consequences  in  failing  to  address  these 
issues.  Although  LMP  became  operational  4  years  ago,  it  continues  to 
encounter  operational  problems.  While  there  have  been  some 
improvements  in  LMP,  key  functionality,  such  as  accurately  billing 
customers  and  recognizing  revenue,  remains  problematic.  The  Army’s 
inability  to  resolve  these  persistent  LMP  problems  can,  in  part,  be 
attributed  to  inadequate  system  testing.  Until  LMP  implements  an  effective 
testing  process,  it  will  have  little  assurance  that  the  development  and 
corrective  actions  it  takes  (1)  are  properly  developed,  and  (2)  do  not 
introduce  additional  defects  into  the  system. 


Recommendations  for 
Executive  Action 


To  improve  the  department’s  efforts  to  achieve  TAV  and  further  enhance 
its  efforts  to  improve  the  control  and  accountability  over  business  system 
investments,  we  recommend  that  the  Secretary  of  Defense  direct  the 
Secretary  of  the  Army  and  the  Director,  BTA,  to  jointly  take  the  following 
five  actions: 


•  Develop  a  concept  of  operations  that  (1)  clearly  defines  the  ERP  vision  for 
accomplishing  total  asset  visibility  within  the  Army;  (2)  addresses  how  its 
business  systems  and  processes,  individually  and  collectively,  will  provide 
the  desired  functionality  to  achieve  TAV;  and  (3)  determines  the  desired 
functionality  among  the  selected  systems. 

•  Develop  policies,  procedures,  and  processes  to  support  the  oversight  and 
management  of  selected  groupings  of  business  systems  that  are  intended 
to  provide  a  specific  capability  or  functionality,  such  as  TAV  from  a 
portfolio  perspective,  utilizing  indicators  such  as  costs,  schedule, 
performance,  and  risks. 

•  Establish  an  IV&V  function  for  GFEBS,  GCSS-Army,  and  LMP. 
Additionally,  direct  that  all  IV&V  reports  for  each  system  be  provided  to 
Army  management,  the  appropriate  IRB,  and  BTA. 

•  Require  that  any  future  GFEBS  economic  analysis  identify  costs  and 
benefits  in  accordance  with  the  criteria  specified  by  DOD  and  OMB 
guidance,  to  include  a  sensitivity  analysis. 
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•  Direct  that  LMP  utilize  systems  testers  that  are  independent  of  the  LMP 
system  developers  to  help  ensure  that  the  system  is  providing  the  users  of 
the  system  the  intended  capabilities. 


We  received  written  comments  on  a  draft  of  this  report  from  the  Deputy 
Under  Secretary  of  Defense  (Business  Transformation),  which  are 
reprinted  in  appendix  II.  Overall,  DOD  concurred  with  our 
recommendations  and  stated  that  it  would  work  diligently  to  implement 
them.  The  comments  included  two  sets  of  specific  responses  to  our 
recommendations — one  set  provided  by  BTA  and  another  provided  by  the 
Army.  In  regard  to  the  response  provided  by  the  Army,  it  concurred  with 
each  of  the  recommendations.  BTA  stated  that  it  fully  agreed  with  our 
observations.  BTA,  though,  partially  concurred  with  all  the 
recommendations  on  the  basis  that  they  were  directed  jointly  to  the 
Secretary  of  the  Army  and  the  Director,  BTA.  BTA  stated  that  under  U.S. 
Code  Title  10  and  in  accordance  with  DOD’s  tiered  accountability,  the 
Army  has  complete  authority  for  execution  of  its  programs  and 
responsibility  for  implementing  the  recommendations.  BTA’s  comments 
noted  that  it  has  neither  the  authority  nor  the  responsibility  to  direct  the 
actions  of  the  Army. 

We  appreciate  the  department’s  willingness  to  address  our 
recommendations.  With  regard  to  BTA’s  concern,  our  recommendations 
do  not  direct  BTA  to  oversee  or  direct  the  Army.  Rather,  the 
recommendations  stated  that  the  specific  actions  should  be  undertaken 
jointly  at  the  direction  of  the  Secretary  of  Defense.  We  continue  to  believe 
that  a  cooperative  and  effectively  coordinated  BTA  and  Army  approach  to 
addressing  our  recommendations  is  most  likely  to  achieve  the 
fundamental  business  system  transformation  necessary  to  achieve  the 
department’s  TAV  objective.  Further,  the  involvement  of  the  Office  of  the 
Secretary  of  Defense,  including  BTA,  in  the  Army’s  investment 
management  practices  is  consistent  with  10  U.S.C.  §§  186  and  2222,  which 
provide  for  DOD  policymaking  and  oversight  of  Army  business  system 
functions. 

In  regard  to  the  responses  provided  by  the  Army,  it  acknowledged  the 
importance  of  a  concept  of  operations  in  achieving  its  goal  of  TAV  and  in 
optimizing  its  business  system  investments.  Further,  the  Army 
acknowledged  that  there  are  risks  involved  in  attempting  to  effectively 
integrate  the  various  system  efforts.  Going  forward,  as  we  recommended, 
it  will  be  important  that  the  Army  develop  a  clear,  long-term  strategy  to 
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help  ensure  that  time  frames,  results-oriented  performance  measures,  and 
accountability  mechanisms  are  established  and  monitored  to  achieve  TAV. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense;  Acting 
Secretary  of  the  Army;  Under  Secretary  of  Defense  (Acquisition, 
Technology,  and  Logistics);  Under  Secretary  of  Defense  (Comptroller); 
Deputy  Under  Secretary  of  Defense  (Business  Transformation);  Deputy 
Under  Secretary  of  Defense  (Financial  Management);  Assistant  Secretary 
of  Defense  (Networks  and  Information  Integration);  Assistant  Secretary  of 
the  Army  (Acquisition,  Logistics,  and  Technology);  Assistant  Secretary  of 
the  Army  (Financial  Management  and  Comptroller);  Army  Chief 
Information  Officer;  Army  Deputy  Chief  of  Staff  (Logistics);  Commander, 
U.S.  Army  Materiel  Command;  and  other  interested  congressional 
committees  and  members.  Copies  of  this  report  will  be  made  available  to 
others  upon  request.  In  addition,  this  report  is  available  at  no  charge  on 
the  GAO  Web  site  at  http://www.gao.gov. 
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Please  contact  McCoy  Williams  at  (202)  512-9095  or  williamsml@gao.gov, 
Keith  A.  Rhodes  at  (202)  512-6412  or  rhodesk@gao.gov,  or  William  M.  Solis 
at  (202)  512-8365  or  solisw@gao.gov  if  you  or  your  staff  have  questions  on 
matters  discussed  in  this  report.  Contact  points  for  our  Offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
of  this  report.  Key  contributors  to  this  report  are  listed  in  appendix  V. 


McCoy  Williams 
Director 

Financial  Management  and  Assurance 


Keith  A.  Rhodes 
Chief  Technologist 
Applied  Research  and  Methods 
Center  for  Technology  and  Engineering 


William  M.  Solis 
Director 

Defense  Capabilities  and  Management 
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Appendix  I:  Scope  and  Methodology 


To  determine  whether  the  Army  has  developed  a  business  system  strategy 
for  achieving  total  asset  visibility  (TAV),  we  met  with  program  office 
officials  for  the  General  Fund  Enterprise  Business  System  (GFEBS),  the 
Global  Combat  Support  System-Army  (GCSS-Army),  and  the  Logistics 
Modernization  Program  (LMP).  We  obtained  briefings  on  the  intended 
purpose  of  each  system  and  walkthroughs  of  various  scenarios  that 
described  how  the  systems  would  transmit  data  between  them.  We  also 
met  with  officials  from  the  Office  of  Program  Analysis  and  Evaluation,  the 
Office  of  the  Assistant  Secretary  of  Defense  (Network  and  Information 
Integration)/the  Department  of  Defense  (DOD)  Chief  Information  Officer, 
and  the  Business  Transformation  Agency  (BTA)  to  identify  and  discuss 
any  issues/concerns  they  have  related  to  the  implementation  of  GFEBS 
and  GCSS-Army.  Furthermore,  we  reviewed  reports  by  the  Army  Audit 
Agency  and  the  DOD  Inspector  General  to  ascertain  if  they  had  previously 
reported  upon  system  weaknesses  related  to  the  three  systems  and  how 
those  weaknesses  would  affect  the  Army’s  vision  of  an  integrated  systems 
strategy. 

In  order  to  determine  if  the  Army  has  effectively  implemented  a 
governance  structure  to  oversee  and  manage  its  business  system 
investment  in  accordance  with  DOD  guidance,  we  reviewed  guidance 
issued  by  the  Office  of  the  Secretary  of  Defense,  the  Army,  and  BTA 
related  to  investment  management.  We  also  obtained  an  understanding  of 
the  Army’s  process  to  comply  with  the  certification  and  annual  system 
review  processes  required  by  the  fiscal  year  2005  National  Defense 
Authorization  Act.1  We  also  obtained  and  analyzed  documentation  related 
to  the  certification  and  annual  system  review  process.  Furthermore,  we 
reviewed  the  Selected  Capital  Investment  Review  reports  as  of  February 
2006  and  February  2007  to  ascertain  if  there  were  any  cost,  schedule,  and 
performance  issues  discussed  in  these  reports  for  the  GFEBS,  GCSS-Army, 
and  LMP.  We  also  reviewed  Defense  Acquisition  Executive  Summary 
reports  for  GFEBS  to  ascertain  if  there  were  any  cost,  schedule,  and 
performance  issues  discussed  in  these  reports.  Similar  reports  were  not 
required  for  GCSS-Army  and  LMP.  Additionally,  we  met  with  BTA  officials 
to  obtain  an  understanding  of  how  the  Enterprise  Risk  Assessment  Model 
is  to  be  utilized  as  part  of  the  department’s  investment  management 
criteria. 


Donald  W.  Reagan  National  Defense  Authorization  Act  for  Fiscal  Year  2005,  Pub.  L.  No. 
108-375,  §  332,  118  Stat.  1811,  1851-56  (Oct.  28,  2004)  (codified,  in  part,  at  10  U.S.C.  §§  186, 
2222). 
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Appendix  I:  Scope  and  Methodology 


To  evaluate  the  extent  to  which  the  Army  has  made  progress  in  correcting 
the  previously  reported  problems  regarding  LMP’s  implementation,  we 
met  with  officials  from  the  LMP  program  office  and  the  Computer 
Sciences  Corporation  (CSC).  Briefings  were  provided  that  detailed  the 
specific  corrective  actions  taken  in  response  to  the  various  issues 
discussed  in  our  two  previous  reports.2  To  substantiate  the  corrective 
actions  taken  to  address  the  accounting  issues,  we  reviewed  the  Defense 
Finance  and  Accounting  Service  (DFAS)  1307  accounting  reports3  and 
discussed  the  issues  detailed  in  the  reports  with  DFAS  personnel 
responsible  for  preparing  the  reports.  We  also  reviewed  the  fiscal  year 
2006  Army  Working  Capital  Fund  Financial  Statements  to  ascertain  the 
specific  issues  related  to  LMP.  Further,  we  met  with  DOD  Inspector 
General  personnel  and  reviewed  their  workpapers  to  obtain  an 
understanding  of  the  deficiencies  discussed  in  the  fiscal  year  2006  Army 
Working  Capital  Fund  Financial  Statements. 

To  assess  whether  the  Army  had  established  and  implemented  disciplined 
processes  related  to  requirements  management  and  testing  we: 

•  obtained  an  understanding  of  the  Army’s  revised  procedures  for  defining 
requirements  management  frameworks  and  compared  these  procedures  to 
its  current  practices; 

•  reviewed  guidance  published  by  the  Institute  of  Electrical  and  Electronics 
Engineers  and  the  Software  Engineering  Institute  and  publications  by 
experts  to  determine  the  attributes  that  should  be  used  for  developing 
good  requirements;  and 

•  selected  10  requirements  and  performed  an  in-depth  review  and  analysis 
to  determine  whether  they  had  the  attributes  normally  associated  with 
good  requirements  and  whether  these  requirements  traced  between  the 
various  process  documents.  These  requirements  were  to  have  followed  the 
revised  requirements  management  process. 


2GAO,  DOD  Business  Systems  Modernization:  Billions  Continue  to  Be  Invested  with 
Inadequate  Management  Oversigh  t  and  Accountability,  GAO-04-615  (Washington,  D.C.: 
May  27,  2004)  and  Army  Depot  Maintenance:  Ineffective  Oversight  of  Depot  Maintenance 
Operations  and  System  Implementation  Efforts,  GAO-05-441  (Washington,  D.C.:  June  30, 
2005). 

3We  reviewed  the  1307  reports  for  March  2004,  September  2004,  March  2005,  September 
2005,  March  2006,  September  2006,  December  2006,  and  February  2007. 
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Appendix  I:  Scope  and  Methodology 


To  augment  these  document  reviews  and  analyses,  we  discussed  our 
analysis  with  LMP  program  office  and  CSC  officials.  We  also  met  with 
officials  from  the  Office  of  Program  Analysis  and  Evaluation,  the  Office  of 
the  Assistant  Secretary  of  Defense  (Network  and  Information 
Integration)/DOD  Chief  Information  Officer  (CIO),  and  the  Business 
Transformation  Agency  to  identify  any  issues/concems  they  have  related 
to  the  implementation  of  LMP. 

We  performed  our  audit  work  from  May  2006  through  June  2007  in 
accordance  with  U.  S.  generally  accepted  government  auditing  standards. 
We  conducted  our  work  at  the  Office  of  the  Assistant  Secretary  of  Defense 
(Networks  and  Information  Integration)/DOD  Chief  Information  Officer; 
the  U.S.  Army  Materiel  Command;  the  Army  CIO;  and  the  Business 
Transformation  Agency.  We  visited  the  Army  Program  Executive  Office 
Enterprise  Information  Systems  and  GFEBS  program  office  at  Ft.  Belvoir, 
Virginia.  We  also  visited  the  Army  contractor’s  site  for  LMP,  Computer 
Sciences  Corporation  in  Marlton,  New  Jersey — and  for  GCSS-Army — 
Northrop  Grumman  in  Chester,  Virginia.  We  also  visited  the  Tobyhanna 
Army  Depot,  Tobyhanna,  Pennsylvania  and  DFAS  Indianapolis,  Indiana. 
We  requested  comments  on  a  draft  of  this  report  from  the  Secretary  of 
Defense  or  his  designee.  We  received  written  comments  from  the  Deputy 
Under  Secretary  of  Defense  (Business  Transformation),  which  are 
reprinted  in  appendix  II. 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


Note:  GAO  comments 
supplementing  those  in 
the  report  text  appear  at 
the  end  of  this  appendix. 


See  comment  1 . 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION, 

TECHNOLOGY 

AND  LOGISTICS  JUL  6  2007 

Mr.  McCoy  Williams 

Director,  Financial  Management  and  Assurance 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Williams: 


This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report  07- 
860,  “DOD  BUSINESS  TRANSFORMATION:  Lack  of  an  Integrated  Strategy  Puts 
the  Army’s  Asset  Visibility  System  Investments  at  Risk”,  dated  June  6,  2007  (GAO 
Code  195087). 


DoD  hereby  acknowledges  receipt  of  the  draft  report.  We  have  reviewed  the 
draft  and  appreciate  the  GAO’s  support  in  identifying  areas  of  improvement  as  it  relates 
to  the  Department  of  Army’s  total  asset  visibility.  Overall,  DoD  concurs  with  the 
GAO’s  recommendations,  except  where  it  is  recommended  that  the  Business 
Transformation  Agency  (BTA)  jointly  direct  the  Army  to  implement  a  recommendation. 
Under  Title  10  U.S.  Code,  the  BTA  neither  has  the  authority  nor  responsibility  to 
direct  the  actions  of  the  Army.  Attached,  you  will  find  responses  to  these 
recommendations  from  both  the  BTA  and  the  Army. 

Still,  DoD  agrees  with  the  core  of  each  recommendation,  and  we  will  work 
diligently  to  close  them.  Where  appropriate,  the  BTA  will  provide  assistance  and 
support  to  the  Army  in  achieving  this  objective.  Additionally,  it  is  DoD’s  intent,  at  an 
enterprise  level,  to  continue  maturing  our  portfolio  management  policies  and 
procedures  such  that  they  align  with  and  assist  the  Army  in  addressing  the 
recommendations  presented  in  the  draft  report. 


Again,  thank  you  for  your  support.  DoD  strongly  values  our  partnership  with 
the  GAO,  and  we  appreciate  your  continued  strides  to  assist  us  with  our  business 
transformation  efforts.  , 


Brinkley 

Deputy  Under  Secretary  of  Defense 
(Business  Transformation) 
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Appendix  II:  Comments  from  the  Department 
of  Defense 


GAO  Draft  Report  Dated  JUNE  6,  2007 
GAO-07-860  (GAO  code  195087) 


‘DOD  BUSINESS  TRANSFORMATION:  LACK  OF  AN  INTEGRATED  STRATEGY 
PUTS  THE  ARMY’S  ASSET  VISIBILITY  SYSTEM  INVESTMENTS  AT  RISK” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 
PROVIDED  BY  THE  BUSINESS  TRANSFORMATION  AGENCY 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  Business  Transformation  Agency  (BTA), 
to  jointly  develop  a  concept  of  operations  that: 

a.  clearly  defines  the  enterprise  resource  planning  (ERP)  vision  for 
accomplishing  total  asset  visibility  within  the  Army; 

b.  addresses  how  its  business  systems  and  processes,  individually  and 
collectively,  will  provide  the  desired  functionality  to  achieve  total  asset 
visibility  (TAV);  and 

c.  determines  the  desired  functionality  among  the  selected  systems.  (Page 
42/GAO  Draft  Report) 


DOD  RESPONSE:  Partially  Concur 

The  Department  concurs  with  the  Army’s  position.  However,  the  overall  Department 
position  is  “partially  concur”  due  to  the  recommendation  being  directed  jointly  to  the 
Secretary  of  the  Army  and  the  Director,  BTA.  While  we  fully  agree  with  GAO’s 
observations,  under  U.S.  Code  Title  10  and  in  accordance  with  the  Department’s  tiered 
accountability  approach,  the  Army  has  complete  authority  for  execution  of  its  programs 
and  responsibility  for  implementing  the  recommendation.  BTA  will  continue  to  provide 
assistance  and  support  to  the  Army  from  an  enterprise  standpoint  as  needed. 

BTA  has  developed  the  BEA  and  has  provided  specific  recommendations  to  the  Army  in 
addressing  TAV  and  other  related  issues,  and  continues  to  work  with  the  Army  for  the 
integration  of  the  proper  functionality.  BTA  has  also  evaluated  the  deployment  of  the 
ERP  solutions  in  accordance  with  commercial  best  practices  and  recommendations  to  the 
Army  which  are  designed  to  achieve  TAV  and  proper  financial  controls. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  BTA,  to  jointly  develop  policies, 
procedures,  and  processes  to  support  the  oversight  and  management  of  selected  groupings 
of  business  systems  that  are  intended  to  provide  a  specific  capability  or  functionality, 
such  as  TAV  from  a  portfolio  perspective,  utilizing  indicators  such  as  costs,  schedule, 
performance,  and  risks.  (Page  43/GAO  Draft  Report) 
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POD  RESPONSE:  Partially  Concur 

The  Department  concurs  with  the  Army’s  position.  However,  the  overall  Department 
position  is  “partially  concur”  due  to  the  recommendation  being  directed  jointly  to  the 
Secretary  of  the  Army  and  the  Director,  BTA.  While  we  fully  agree  with  GAO’s 
observations,  under  U.S.  Code  Title  10  and  in  accordance  with  the  Department’s  tiered 
accountability  approach,  the  Army  has  complete  authority  for  execution  of  its  programs 
and  responsibility  for  implementing  the  recommendation.  BTA  will  continue  to  provide 
assistance  and  support  to  the  Army  from  an  enterprise  standpoint  as  needed. 

It  is  the  Department’s  intent,  at  an  enterprise  level,  to  continue  maturing  our  portfolio 
management  policies  and  procedures. 

RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  BTA  jointly  establish  an  independent 
verification  and  validation  (IV&V)  function  for  General  Fund  Enterprise  Business 
System  (GFEBS),  Global  Combat  Support  System-Army  Field/Tactical  (GCSS-Army) 
and  Logistics  Modernization  Program  (LMP).  Additionally,  direct  that  all  IV&V  reports 
for  each  system  be  provided  to  Army  management,  the  appropriate  investment  review 
board  (IRB)  and  BTA.  (Page  43/GAO  Draft  Report) 

POD  RESPONSE:  Partially  Concur 

The  Department  concurs  with  the  Army’s  position.  However,  the  overall  Department 
position  is  “partially  concur”  due  to  the  recommendation  being  directed  jointly  to  the 
Secretary  of  the  Army  and  the  Director,  BTA.  While  we  fully  agree  with  GAO’s 
observations,  under  U.S.  Code  Title  10  and  in  accordance  with  the  Department’s  tiered 
accountability  approach,  the  Army  has  complete  authority  for  execution  of  its  programs 
and  responsibility  for  implementing  the  recommendation.  BTA  will  continue  to  provide 
assistance  and  support  to  the  Army  from  an  enterprise  standpoint  as  needed. 

RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Anny  and  the  Director  of  BTA,  to  jointly  require  that  any  future 
GFEBS  economic  analysis  identify  costs  and  benefits  in  accordance  with  the  criteria 
specified  by  DoD  and  OMB  guidance,  to  include  a  sensitivity  analysis.  (Page  43/GAO 
Draft  Report) 

DOD  RESPONSE:  Partially  Concur 

The  Department  concurs  with  the  Army’s  position.  However,  the  overall  Department 
position  is  “partially  concur”  due  to  the  recommendation  being  directed  jointly  to  the 
Secretary  of  the  Army  and  the  Director,  BTA.  While  we  fully  agree  with  GAO’s 
observations,  under  U.S.  Code  Title  10  and  in  accordance  with  the  Department’s  tiered 
accountability  approach,  the  Army  has  complete  authority  for  execution  of  its  programs 
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and  responsibility  for  implementing  the  recommendation.  BTA  will  continue  to  provide 
assistance  and  support  to  the  Army  from  an  enterprise  standpoint  as  needed. 

RECOMMENDATION  5:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  BTA,  to  jointly  direct  the  LMP  utilize 
system  testers  that  are  independent  of  the  LMP  system  developers  to  help  ensure  that  the 
system  is  providing  the  users  of  the  system  the  intended  capabilities.  (Page  43/GAO 
Draft  Report) 

POD  RESPONSE:  Partially  Concur 

The  Department  concurs  with  the  Army’s  position.  However,  the  overall  Department 
position  is  “partially  concur”  due  to  the  recommendation  being  directed  jointly  to  the 
Secretary  of  the  Army  and  the  Director,  BTA.  While  we  fully  agree  with  GAO’s 
observations,  under  U.S.  Code  Title  10  and  in  accordance  with  the  Department’s  tiered 
accountability  approach,  the  Army  has  complete  authority  for  execution  of  its  programs 
and  responsibility  for  implementing  the  recommendation.  BTA  will  continue  to  provide 
assistance  and  support  to  the  Army  from  an  enterprise  standpoint  as  needed. 
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GAO  Draft  Report  Dated  JUNE  6,  2007 
GAO-07-860  (GAO  code  195087) 


“DOD  BUSINESS  TRANSFORMATION:  LACK  OF  AN  INTEGRATED  STRATEGY 
PUTS  THE  ARMY’S  ASSET  VISIBILITY  SYSTEM  INVESTMENTS  AT  RISK” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 
PROVIDED  BY  THE  ARMY 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  Business  Transformation  Agency  (BTA), 
to  jointly  develop  a  concept  of  operations  that: 

a.  clearly  defines  the  enterprise  resource  planning  (ERP)  vision  for 
accomplishing  total  asset  visibility  within  the  Army; 

b.  addresses  how  its  business  systems  and  processes,  individually  and 
collectively,  will  provide  the  desired  functionality  to  achieve  total  asset 
visibility  (TAV);  and 

c.  determines  the  desired  functionality  among  the  selected  systems.  (Page 
42/GAO  Draft  Report) 


DOD  RESPONSE:  Concur 

•  The  Army  is  amenable  to  ideas  that  will  facilitate  our  optimization  goals  to 
achieve  TAV.  We  believe  that  a  concept  of  operations  that  includes  the  three 
components  described  in  Recommendation  1  will  help  us  attain  our  optimization 
goals.  We  have  blueprinted  our  systems  based  on  requirements  and  priorities  and 
divided  that  functionality  among  the  three  ERP  systems.  Above  all,  our  goal  is 
an  efficient  and  effective  deployment  of  automated  business  capabilities  given  the 
realities  of  where  the  Army  is  today,  strategically,  financially,  and 
programmatically. 

•  The  Army  agrees  that  we  should  optimize  our  IT  business  investments  to  achieve 
the  goal  of  TAV.  At  program  initiation,  the  Army  functionally  aligned  its 
financial  and  logistics  ERP  solutions.  We  further  recognize  the  synchronization 
risk  between  ERP  programs  and  manage  accordingly.  Hence,  the  Army  has  a 
single  materiel  developer  organization,  PEO  Enterprise  Information  Systems 
(PEO  EIS),  and  a  single  manager  to  whom  the  ERP  Product  Managers  report.  A 
recent  realignment  of  the  Logistics  Modernization  Program  (LMP)  from  an  Army 
Command  to  the  PEO  EIS  facilitated  unity  of  purpose,  helping  to  synchronize  the 
acquisition  aspects  of  a  solution  that  enables  TAV  for  the  Army.  Moreover,  we 
have  established  a  functional  cross-domain  working  group  between  the  Single 
Automated  Logistics  Enterprise  (SALE)  and  the  Single  Automated  Financial 
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Enterprise  (SAFE)  that  meets  regularly  to  decide  optimal  implementations  of 
business  processes  to  achieve  visibility  of  Army  assets  that  transcend  Domains 
and  the  tactical  and  national  levels  of  logistics. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  BTA,  to  jointly  develop  policies, 
procedures,  and  processes  to  support  the  oversight  and  management  of  selected  groupings 
of  business  systems  that  are  intended  to  provide  a  specific  capability  or  functionality, 
such  as  TAV  from  a  portfolio  perspective,  utilizing  indicators  such  as  costs,  schedule, 
performance,  and  risks.  (Page  43/GAO  Draft  Report) 

POD  RESPONSE:  Concur. 

While  we  are  an  Army  at  war,  men  and  women  are  working  diligently  to  develop  and 
institutionalize  an  oversight  system  for  business  systems  that  is  austere,  yet  effective; 
defined  and  repeatable  but  not  bureaucratic;  and  focused  on  the  Army’s  ability  to  man, 
equip,  train,  and  maintain  our  Forces  in  the  field.  In  this  regard,  the  Army  Business 
Mission  Area  has  allocated  resources  to  assist  in  the  maturation  of  Army  Portfolio 
Management  (PfM). 

RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  BTA  jointly  establish  an  independent 
verification  and  validation  (IV&V)  function  for  General  Fund  Enterprise  Business 
System  (GFEBS),  Global  Combat  Support  System-Army  Field/Tactical  (GCSS-Army) 
and  Logistics  Modernization  Program  (LMP).  Additionally,  direct  that  all  IV&V  reports 
for  each  system  be  provided  to  Army  management,  the  appropriate  investment  review 
board  (IRB)  and  BTA.  (Page  43/GAO  Draft  Report) 

POD  RESPONSE:  Concur. 

As  previously  stated,  the  Army  has  charged  the  PEO  EIS  to  manage  the  cost,  schedule, 
and  performance  aspects  of  the  GFEBS,  GCSS-Army,  and  LMP.  The  organization  is 
governed  by  and  accountable  to  the  Milestone  Decision  Authority  according  to  the 
program  baselines  established.  The  Army  recognizes  the  need  for  a  enterprise  view  of 
the  risks  associated  with  each  program.  Therefore,  the  PEO  EIS  will  ensure 
establishment  of  an  Independent  Validation  and  Verification  (IV&V)  process  for  each 
program  that  provides  for  risk  management  at  a  level  where  management  responsibility 
exists  for  each  of  the  three  ERP  systems  and  independent  of  the  programs.  All  PEO  EIS 
IV&V  results  will  be  provided  to  the  Department  of  the  Army  and  relevant  management 
organizations  as  required. 


RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  BTA,  to  j  ointly  require  that  any  future 
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See  comment  2. 


See  comment  3. 


GFEBS  economic  analysis  identify  costs  and  benefits  in  accordance  with  the  criteria 
specified  by  DoD  and  OMB  guidance,  to  include  a  sensitivity  analysis.  (Page  43/GAO 
Draft  Report) 

DOD  RESPONSE:  Concur. 

•  The  GAO  report  cites  S142M  of  questionable  benefits,  comprised  of  "transfers". 
While  the  economics  of  the  transfers  was  debated  with  GAO,  the  real  point  should 
be  that  these  are  S142M  questionable  benefits  of  S6,667M  total  benefits.  GAO 
takes  issue  with  not  adequately  justifying  2.1%  of  the  benefits.  Assuming  that  the 
2.1%  is  not  legitimate,  eliminating  these  benefits  at  the  very  worst  lowers  the 
Benefit-Cost  Ratio  from  2.35  to  2.28,  which  still  would  justify  the  investment. 

•  The  GAO  report  cites  the  lack  of  a  sensitivity  analysis.  The  Cost  Review  Board 
Working  Group  included  the  sensitivity  analysis  by  adding  into  the  estimate  the 
uncertainty  surrounding  the  costs  of  interfaces.  While  this  does  not  show  up  as  a 
titled  section  in  the  EA  called  "Sensitivity",  the  results  of  the  analysis  were  in  fact 
included  in  the  GFEBS  EA. 

RECOMMEND  ATION  5:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Secretary  of  the  Army  and  the  Director  of  BTA,  to  jointly  direct  the  LMP  utilize 
system  testers  that  are  independent  of  the  LMP  system  developers  to  help  ensure  that  the 
system  is  providing  the  users  of  the  system  the  intended  capabilities.  (Page  43/GAO 
Draft  Report) 

DOD  RESPONSE:  Concur. 
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The  following  are  GAO’s  comments  on  DOD’s  letter  dated  July  6,  2007. 


GAO  Comments 


1.  See  the  “Agency  Comments  and  Our  Evaluation”  section  of  this  report. 

2.  In  regard  to  the  GFEBS  economic  analysis,  the  program  office  did  not 
provide  the  necessary  support  for  us  to  replicate  the  estimated  program 
savings. 

3.  In  regard  to  the  sensitivity  analysis,  the  Army’s  approach  of  adding  the 
uncertainty  surrounding  the  costs  of  interfaces  into  the  estimate  is  not 
consistent  with  OMB,  DOD,  and  Army  guidance,  as  discussed  in  the  report. 
Rather,  a  sensitivity  analysis  is  accomplished  by  changing  the  numerical 
value  of  a  given  variable  to  gauge  the  effect  of  that  change  on  model 
results,  such  as  the  benefit-to-cost  ratio.  A  sensitivity  analysis  identifies 
key  assumptions  and  varies  a  single  assumption  while  holding  the  others 
constant  to  determine  what  amount  of  change  in  that  assumption  is 
required  to  raise  or  lower  the  resulting  dominant  benefit  or  cost  estimates 
by  a  set  amount. 
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Fundamental  to  successful  enterprise  resource  planning  (ERP) 
implementation  is  the  reengineering  of  an  organization’s  business 
processes  in  a  manner  that  helps  ensure  that  the  right  resources  are 
available  when  needed.  While  the  Army  has  stated  that  through  the 
successful  implementation  of  the  General  Fund  Enterprise  Business 
System  (GFEBS),  the  Global  Combat  Support  System-Army  (GCSS-Army), 
and  the  Logistics  Modernization  Program  (LMP)  will  reengineer  its 
business  processes,  we  found  some  business  processes  that  have  been 
adopted  or  are  expected  to  be  adopted  have  not  embraced  the  basic 
concept  of  an  ERP  solution.  Details  concerning  the  Army’s  strategy  for 
handling  funds  control  and  disbursements  are  highlighted  below. 


Funds  Control 


The  funds  control  concept  that  will  be  utilized  by  GCSS-Army  and  GFEBS 
is  that  all  funds  control  activities  will  be  handled  by  GFEBS.  Accordingly, 
an  interface  must  be  developed  to  transmit  the  appropriate  information 
between  the  two  systems  in  order  to  ensure  that  all  of  the  information  is  in 
agreement.  Figure  6  illustrates  how  the  process  is  intended  to  work. 
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Figure  6:  Planned  GFEBS  and  GCSS-Army  Fund  Control  Process 


Source:  Army. 


As  shown  in  figure  6,  information  will  have  to  be  transmitted  from  GCSS- 
Army  each  and  every  time  there  is  a  transaction  that  is  associated  with 
funds  control.  This  approach  does  not  take  full  advantage  of  the 
capabilities  within  the  ERP  software  that  will  be  utilized  by  GCSS-Army. 
Figure  7  shows  an  example  of  more  efficient  and  streamlined  funds 
control  process  based  on  ERP  concepts  that  eliminates  these  various 
transactions  having  to  be  processed  between  GFEBS  and  GCSS-Army. 
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Figure  7:  GCSS-Army  Funds  Control  Process  Utilizing  ERP  Capabilities 


Organizations  responsible  for  funds  control 
(for  example,  Army  CFO)  perform  funds  control 
responsibilities  (including  establishing  limits  and 
reallocating  funds)  through  access  to  GCSS-Army 


Source:  Army. 


We  were  informed  by  GFEBS  program  officials  that  funds  control  for  the 
activities  associated  with  GCSS-Army  needed  to  be  maintained  in  GFEBS 
because  (1)  GCSS-Army  was  not  considered  an  official  accounting  system, 
(2)  the  version  of  the  software  initially  selected  by  GCSS-Army  could  not 
meet  the  requirements  of  the  Federal  Financial  Management  Improvement 
Act  of  1996  (FFMLA),1  and  (3)  the  Army  wanted  to  centrally  manage 
general  fund  activities.  In  effect,  the  Army’s  approach  mirrors  the 
functionality  in  the  stovepiped  processes  of  the  legacy  systems,  dedicated 
processes  to  obtain  information  for  specific  reasons  rather  than  ensuring  a 
given  process  provided  the  information  needed  from  an  enterprise  point  of 
view.  Furthermore,  the  concern  that  the  version  of  software  being  used  by 
GCSS-Army  had  not  been  certified  as  compliant  with  federal  requirements 


'Pub.  L.  No.  104-208,  div.  A,  §101(F),  title  VIII,  110  Stat.  3009,  3009-389  (Sept.  30,  1996). 
FFMIA  requires  the  24  Chief  Financial  Officers  Act  departments  and  agencies  to  implement 
and  maintain  financial  management  systems  that  comply  substantially  with  (1)  federal 
financial  management  system  requirements,  (2)  applicable  federal  accounting  standards, 
and  (3)  the  U.S.  Government  Standard  General  Ledger  at  the  transaction  level. 
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can  be  easily  addressed.  Army  officials  have  stated  that  the  software 
vendor  is  committed  to  obtaining  the  necessary  certifications  and  expects 
that  these  will  be  granted  before  GCSS-Army  is  deployed.  From  a 
corporate  perspective,  whether  one  system  or  two  systems  is  the  proper 
approach  is  not  necessarily  the  question  that  the  Army  needs  to  answer. 
Rather,  the  Army  needs  to  address  how  best  to  utilize  the  capabilities  of 
the  ERP  to  achieve  an  effective  funds  control  system. 


Disbursements 


The  Department  of  Defense  (DOD)  Inspector  General  recently  reported 
that  LMP  has  been  implemented  without  the  functionality  to  match 
proposed  disbursements  with  corresponding  obligations  before  making 
any  payments.2  This  process  is  referred  to  as  prevalidation,  which  was 
mandated  with  the  passage  of  the  fiscal  year  1995  Department  of  Defense 
Appropriations  Act3  because  of  DOD’s  long-standing  difficulty  in  properly 
matching  disbursements  with  corresponding  obligations.  Implementing 
LMP  without  this  functionality  is  an  example  of  suboptimizing  the  system, 
resulting  in  the  failure  to  address  a  long-standing  weakness  in  matching 
obligations  with  proposed  disbursements  for  the  Army  Working  Capital 
Fund.  This  functionality  is  part  of  the  basic  software  package  that  is  being 
used  by  LMP.  We  noted  that  the  Army  expects  to  use  the  same  ERP 
software  package  for  GFEBS. 

The  DOD  Inspector  General  reported  that,  rather  than  reengineering  its 
business  processes,  the  Army  decided  to  follow  its  existing  business 
processes  that  were  inefficient  and  precluded  the  Army  from  being  in 
compliance  with  the  fiscal  year  1995  legislative  requirement.  As  a  result  of 
this  decision,  the  Army  has  not  taken  full  advantage  of  the  capabilities 
within  the  ERP  software  package  being  used  to  implement  LMP. 
Considering  that  the  Army  has  already  invested  approximately 
$452  million  in  LMP,  and  as  previously  noted  is  expecting  to  invest 
hundreds  of  million  more,  taking  full  advantage  of  its  capabilities  is  also  a 
prudent  financial  decision.  Because  the  Army  failed  to  reengineer  its 
business  processes,  it  continues  to  use  an  off-line  database,  managed  by 
the  Defense  Finance  and  Accounting  Service  (DFAS),  to  compare 
entitlement  and  accounting  data  and  provide  approval  for  prevalidation 
requests.  The  DOD  Inspector  General  report  noted  that  the  database 


“DOD  Inspector  General,  Controls  Over  the  Prevalidation  of  DOD  Commercial  Payments, 
Report  Number  D2007-065  (Arlington,  Va.:  Mar.  2,  2007). 

3Pub.  L.  No.  103-335,  §  8137,  108  Stat.  2599,  2654-55  (Sept.  30,  1994). 
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contained  errors  in  the  accounting  data,  which  resulted  in  prevalidation 
failures  and  the  need  for  additional  research  and  rework  by  DFAS. 

A  more  efficient  and  streamlined  process  could  be  achieved  by  adopting 
ERP  processes  for  disbursements.  For  example,  by  utilizing  ERP  concepts 
LMP  would  be  the  accounting  system  of  record  and  responsible  for 
ensuring  that  the  applicable  federal  requirements  governing  fund  control 
and  disbursements  were  effectively  implemented.  Further,  the 
disbursements  would  be  made  directly  by  the  Department  of  the 
Treasury — much  like  the  process  used  by  civilian  agencies — rather  than 
going  through  DFAS.  We  recognize  that  this  approach  involves  some 
technical  issues,  such  as  building  the  necessary  interfaces  with  the 
Department  of  the  Treasury.  However,  the  commercial-off-the-shelf 
(COTS)  package  adopted  by  LMP  is  already  required  by  its  Office  of 
Management  and  Budget  certification  to  provide  the  necessary  support  for 
these  interfaces.  The  elimination  of  the  DFAS  interfaces,  which  are  not 
inherently  supported  by  the  COTS  package,  should  help  reduce  the  cost 
and  risk  associated  with  this  change. 
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The  Tobyhanna  Army  Depot  continues  to  experience  financial 
management  problems  because  significant  operational  and  developmental 
issues  related  to  the  Logistics  Modernization  Program  (LMP)  have  not 
been  resolved.  More  specifically,  LMP  continues  to  experience  problems 
with  accurately  recognizing  revenue  and  billing  customers.  This  problem 
has  existed  virtually  since  the  implementation  of  LMP  in  July  2003.  While 
the  Army  and  its  contractor  have  made  numerous  attempts  to  fix  the 
problem,  they  have  not  been  successful.  While  we  found  that  the 
requirements  management  processes  have  improved — a  critical  first  step 
in  reducing  risk  to  acceptable  levels1 — we  continue  to  have  reservations 
about  the  adequacy  of  LMP’s  testing. 


Billing  Problems 
Continue  to  Plague 


LMP 


LMP’s  continuing  billing  problems  have  resulted  in  (1)  customers  not 
being  billed  for  costs  incurred  that  should  have  been  billed  and 
(2)  customers  being  billed  too  much.  According  to  information  provide  by 
the  LMP  program  office,  146  customer  orders  valued  at  approximately 
$5.4  million  were  not  billed  during  the  January  31,  2007,  billing  cycle. 
Customer  orders  that  were  not  recognized  as  revenue  and  billed  represent 
funds  that  Tobyhanna  Army  Depot  is  entitled  to,  because  the  work 
requested  by  the  customer  was  performed.  It  is  essential  that  revenue  is 
properly  recognized  and  customers  billed  for  work  performed  because  it  is 
the  means  by  which  the  depot  finances  its  day-to-day  operations.  Similarly, 
during  the  January  31,  2007,  billing  cycle  about  308  customer  orders 
amounting  to  about  $5.8  million  were  shown  as  being  overbilled  and  a 
credit  should  have  been  issued  to  the  customer.  It  is  critical  that  the  credit 
be  provided  to  the  customer  as  soon  as  possible  since  the  customer  may 
be  able  to  use  these  funds  to  meet  other  funding  demands. 

Our  analysis  of  the  January  2007  billings  for  Tobyhanna  disclosed  that,  of 
the  146  unbilled  customer  orders,  61  (about  42  percent)  related  to  fiscal 
years  2006  and  2007.  Similarly,  for  the  credits,  178  of  308  customer  orders 
(about  58  percent)  were  for  fiscal  years  2006  and  2007.  From  a  dollar 
perspective,  most  of  these  billings  also  related  to  fiscal  years  2006  and 
2007  transactions.  The  fact  that  these  billing  problems  have  persisted 
clearly  indicates  that  the  various  efforts  by  the  Army  to  resolve  this  critical 


Acceptable  levels  refer  to  the  fact  that  any  systems  acquisition  effort  will  have  risks  and 
will  suffer  the  adverse  consequences  associated  with  defects  in  the  processes.  However, 
effective  implementation  of  disciplined  processes  reduces  the  possibility  of  the  potential 
risks  actually  occurring  and  prevents  significant  defects  from  materially  affecting  the  cost, 
timeliness,  and  performance  of  the  project. 
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aspect  of  the  system  have  failed.  Table  1  shows  the  dollar  value  and  the 
number  of  customer  orders  related  to  unbilled  work  and  credits  for  3 
months — November  and  December  2006  and  January  2007. 


Table  1:  Unbilled  and  Credit  Customer  Orders  at  the  Tobyhanna  Army  Depot  for 


November  2006  through  January  2007 

Month 

Unbilled 

customer 

orders 

Dollar  value 
of  unbilled 
customer  orders 

Customer 
orders  with 
credits 

Dollar  value 
of  customer 
orders  with 
credits 

November  2006 

84 

(3,161,490) 

291 

3,156,533 

December  2006 

92 

(2,637,903) 

301 

4,772,884 

January  2007 

146 

(5,367,798) 

308 

5,792,851 

Source:  LMP  program  office. 

The  continuing  billing  problem  at  Tobyhanna  is  also  a  factor  contributing 
to  higher  depot  bills  from  the  Defense  Finance  and  Accounting  Service 
(DFAS) — the  Department  of  Defense’s  (DOD)  centralized  finance  and 
accounting  organization.  According  to  DFAS  personnel,  the  recurring 
billing  and  other  problems  have  resulted  in  DFAS  processing  Tobyhanna’s 
bills  manually  and  performing  other  manual  actions  that  are  not  required 
by  the  other  depots  that  do  not  use  LMP.  According  to  the  information 
provided  by  DFAS,  the  number  of  hours  spent  on  DFAS  accounting 
services  billings  for  Tobyhanna  has  increased  between  fiscal  years  2004 
and  2006.  Table  2  shows  the  number  of  hours  and  related  amounts  DFAS 
billed  Tobyhanna  in  each  year. 


Table  2:  DFAS  Billings  to  the  Tobyhanna  Army  Depot  for  Accounting  Services 

Fiscal  year 

Number  of  hours 

Amount  billed  (dollars) 

2004 

17,737 

1,105,168 

2005 

21,197 

1 ,360,407 

2006 

22,653 

1 ,432,777 

Source:  GAO  based  upon  data  provided  by  DFAS. 


Furthermore,  according  to  DFAS  personnel,  while  the  accounting  needs  of 
other  depots  requires  the  support  of  one  full-time  DFAS  staff,  DFAS  uses 
three  full-time  personnel  to  service  Tobyhanna  due  to  the  numerous 
manual  workarounds  associated  with  LMP.  The  manual  workarounds 
required  to  address  LMP’s  billing  issues  results  in  higher  DFAS  bills  for 
Tobyhanna,  which  in  turn  increases  the  prices  charged  by  Tobyhanna  to 
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its  customers  for  the  work  performed.  Since  Tobyhanna  is  part  of  the 
Army  Working  Capital  Fund,  the  costs  incurred  for  accounting  services  are 
passed  on  to  the  customer  in  the  form  of  higher  depot  prices.  During  the 
course  of  our  audit,  LMP  program  officials  stated  that  they  were  finalizing 
efforts  to  revise  the  billing  process.  At  completion  of  our  field  work,  the 
LMP  program  office  stated  that  it  anticipated  having  the  revised  process  in 
place  in  the  August  2007  time  frame. 

Since  billing  and  revenue  recognition  have  been  such  a  long-standing 
issue,  and  given  that  the  entire  process  is  fundamental  to  the  efficient  and 
effective  operation  of  the  Working  Capital  Fund,  it  is  incumbent  upon  the 
LMP  program  office  to  closely  monitor  the  planned  corrective  actions  and 
obtain  reasonable  assurance  that  the  revised  process  is  meeting  the 
Army’s  expectations.  System  testing  is  a  critical  process  that  should  help 
improve  an  entity’s  confidence  that  the  system  will  satisfy  the 
requirements  of  the  end  user  and  will  operate  as  intended.  If  the  planned 
corrective  actions  do  not  resolve  the  existing  problem,  Tobyhanna’s 
operations  will  continue  to  be  adversely  affected.  As  we  have  previously 
recommended,  LMP  should  not  be  deployed  to  any  additional  locations.2 


Accuracy  of  Financial 
Information  Reported 
by  LMP  Still  a 
Concern 


LMP  continues  to  experience  difficulty  in  accurately  reporting  on  the 
results  of  operations.  For  example,  the  DOD  Inspector  General’s  fiscal 
year  2006  financial  statement  audit  of  the  Army  Working  Capital  Fund 
identified  instances  in  which  LMP  was  not  properly  recording  transactions 
in  accordance  with  the  U.S.  Government  Standard  General  Ledger 
requirements.  Further,  our  analysis  of  the  DFAS  monthly  report  on  depot 
operations — commonly  referred  to  as  the  1307  report3 — found  that 
inaccurate  account  balances  continue  to  persist  in  LMP.  These  issues  have 
presented  many  challenges  that  are  reflected  throughout  the  Army 
Working  Capital  Fund  financial  statements  as  adjustments,  corrections, 
and  footnote  disclosures.  More  specific  details  on  the  problems  are 
highlighted  below. 


•  The  DOD  Inspector  General  Report  on  Internal  Controls  and  Compliance 
with  Laws  and  Regulations  noted  that  the  nonfederal  accounts  payable 


2GAO,  Army  Depot  Maintenance:  Ineffective  Oversight  of  Depot  Maintenance  Operations 
and  System  Implementation  Efforts,  GAO-05-441  (Washington,  D.C.:  June  30,  2005). 

3We  reviewed  the  1307  reports  for  March  2004,  September  2004,  March  2005,  September 
2005,  March  2006,  September  2006,  December  2006,  and  February  2007. 
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account  was  misstated  by  approximately  $175  million  at  the  end  of  fiscal 
year  2006.  This  misstatement  arose  because  Army  entities  using  LMP  had 
not  resolved  approximately  $89  million  in  abnormal  accounts  payable 
balances  reported  by  the  Supply  Management  business  area.  An  abnormal 
account  balance  is  one  in  which  the  recorded  amount  has  been  incorrectly 
classified.  During  fiscal  year  2006,  the  LMP  program  office  initiated 
systemic  corrections  to  reduce  the  reported  abnormal  balances.  Our 
analysis  of  the  first  quarter  fiscal  year  2007  financial  statements  noted  that 
the  reported  balance  had  been  reduced  to  approximately  $81  million.  The 
LMP  program  office  was  continuing  to  research  the  issue. 

•  Our  analysis  of  the  DFAS  1307  reports  found  that  abnormal  balances  in 
accounts  payable  have  been  reported  since  fiscal  year  2004.  LMP  program 
officials  stated  that  abnormal  balances  were  primarily  due  to  the  migration 
of  remaining  obligations  on  open  contracts  from  the  legacy  systems. 
Substantial  effort  was  made  in  fiscal  year  2006  to  correct  the  abnormal 
accounts  payable  balances  that  were  reported  by  LMP,  which  resulted  in  a 
reduction  from  $334  million  at  the  end  of  fiscal  year  2005  to  $88  million  at 
the  end  of  fiscal  year  2006.  The  Army’s  efforts  to  clean  up  abnormal 
account  balances  continued  in  fiscal  year  2007. 

The  Army  has  developed  a  specific  plan  to  address  the  financial 
management  problems  recognized  in  these  statements.  For  example,  a 
Tobyhanna  Army  Depot  on-site  support  team  has  been  assembled  to 
complete  the  data  cleanup,  validate  that  all  system  and  procedural  issues 
have  been  identified,  document  business  processes  and  procedures,  and 
train  Tobyhanna  users.  However,  until  these  problems  are  corrected,  LMP 
will  not  be  able  to  provide  reliable  and  accurate  financial  information  to 
the  Congress  or  Army  management. 


Adequacy  of  LMP 
Testing  Continues  to 
Be  a  Concern 


Our  prior  audits* * * 4  of  LMP  identified  significant  weaknesses  with  LMP’s 
efforts  to  effectively  implement  the  requirements  management  and  testing 

processes  needed  to  reduce  risks  to  acceptable  levels.  During  our  current 
audit,  we  found  that  the  requirements  management  processes  have 
improved — a  critical  first  step — but  we  continue  to  have  reservations 
about  the  adequacy  of  the  testing.  LMP  program  officials  stated  that  these 

testing  weaknesses  are  being  addressed  and  expect  that  an  improved 


4GAO,  DOD  Business  Systems  Modernization:  Billions  Continue  to  Be  Invested  with 
Inadequate  Management  Oversight  and  Accountability,  GAO-04-615  (Washington,  D.C.: 
May  27,  2004)  and  GAO-05-441. 
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testing  process  will  be  effectively  implemented  to  reduce  risks  to 
acceptable  levels. 


LMP’s  Requirements  During  the  current  review,  Army  and  contractor  Computer  Sciences 

Management  Practices  Corporation  (CSC)  officials  stated  that  they  have  acted  upon  our  prior 

Have  Improved  recommendations  and  taken  steps  to  improve  the  program’s  requirements 

management  practices.  In  setting  forth  the  revised  requirements 
management  process,  LMP  and  CSC  officials  noted  that  the  new  process 
generally  envisioned  having  several  documents  that  outlined  the 
requirements  that  needed  to  be  addressed  with  each  document  containing 
specificity  needed  for  the  intended  audience. 

To  ascertain  if  the  Army’s  stated  corrective  actions  were  being  adhered  to, 
we  selected  10  requirements  for  detailed  review.  Our  analysis  found  that 
the  LMP  documentation  in  support  of  the  requirements  management 
practices  was  generally  in  compliance  with  the  stated  process.  More 
specifically,  the  requirements  reviewed  were  generally  consistent  between 
the  documents  and  appeared  to  contain  the  necessary  specificity  to  reduce 
requirement-related  defects  to  acceptable  levels.  Our  observations  were 
consistent  with  the  conclusions  reached  by  a  subsequent  review 
performed  by  the  LMP  verification  and  validation  contractor,  who 
examined  the  same  documentation  that  we  analyzed. 


Concerns  regarding  LMP  Our  review  of  the  stated  testing  processes  found  that  many  of  the 
Testing  Processes  Remain  attributes  associated  with  a  disciplined  testing  process  were  present. 

However,  based  upon  our  review  and  analysis  of  the  10  selected  corrective 
actions,  we  still  have  concerns  that  the  LMP  testing  process  may  not  be 
adequate  since  we  identified  specific  weaknesses  in  the  10  corrective 
actions  reviewed.  System  testing  is  a  critical  process  utilized  by 
disciplined  organizations  and  improves  an  entity’s  confidence  that  the 
system  will  satisfy  the  requirements  of  the  end  user  and  operate  as 
intended.  The  stated  testing  process  included  the  following: 

•  Documenting  scenario  testing.  A  scenario  is  a  business  process  that 
typically  consists  of  several  events.  For  example,  a  scenario  could  be 
developed  for  small  purchases,  which  would  include  events  such  as 
(1)  initiating  a  purchase  request,  (2)  approving  the  request,  (3)  obligating 
the  necessary  funds,  (4)  ordering  the  item,  (5)  receiving  the  item,  and 
(6)  paying  for  the  item.  One  benefit  of  testing  scenarios  is  that  it  helps  to 
ensure  that  information  is  accurately  passed  from  one  process  to  another. 
For  instance,  the  denial  of  a  requisition  prevents  the  purchase  of  the  item. 
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•  Linking  requirements  to  the  test  cases.  Linking  requirements  and  test 
cases  helps  ensure  that  either  (1)  all  requirements  are  adequately  tested  or 
(2)  the  risks  associated  with  not  testing  a  requirement  can  be  properly 
evaluated.  Because  it  is  not  economically  feasible  to  develop  a  testing 
program  that  can  find  every  defect,  it  is  critical  that  an  approach  be 
developed  to  ensure  that  testing  resources  are  focused  on  the  areas  of 
highest  risk.  Accordingly,  linking  the  requirements  makes  it  easier  to 
determine  which  combinations  of  items  are  not  tested,  thereby  enabling 
the  Army  to  evaluate  the  risks  of  not  conducting  those  tests. 

•  Regression  testing.  Regression  testing  is  a  process  that  helps  ensure  that 
changes  made  to  the  system  have  not  adversely  affected  functionality  that 
was  working  prior  to  a  change  being  made  to  the  system.  In  essence,  it  is  a 
process  that  retests  the  entire  system  to  ensure  that  the  problem  fixed  did 
not  have  an  adverse  effect  on  other  functions  within  the  system. 

While  the  processes  described  above  are  an  improvement  over  what  we 
have  seen  in  the  past,  we  continue  to  identify  weaknesses  in  the  testing 
process.  For  example,  none  of  the  test  scripts  provided  adequate 
information  that  linked  the  items  in  the  test  script  to  a  specific 
requirement.  This  linking  is  commonly  referred  to  as  traceability  and  is 
characteristic  of  a  disciplined  testing  process.  The  test  scripts  reviewed 
contained  headings  for  information  on  the  scenario,  key  data 
requirements,  expected  results,  actual  results,  and  whether  the  test  was 
considered  successful.  However,  the  actual  test  script  failed  to  contain  the 
level  of  specificity  that  clearly  delineated  how  a  specific  requirement 
identified  in  the  requirement  document(s)  and  the  associated 
requirement(s)  in  that  document  were  being  tested.  As  a  result,  it  is 
virtually  impossible  to  ascertain  if  the  requirement  was  properly  tested. 
For  example,  several  requirements  defined  rules  that  were  expected  to  be 
implemented.  Without  linking  a  given  requirement  to  the  tests  designed  to 
exercise  that  requirement,  it  was  impossible  to  obtain  reasonable 
assurance  that  (1)  all  the  requirements  had  been  tested  and  (2)  the  test 
provided  adequate  coverage  for  each  requirement. 

A  review  of  the  test  script  for  one  corrective  action  showed  that  the 
identified  expected  results  were  “output  of  report  should  show  open 
accounts  receivable  items  for  customers  in  the  range  noted.”  Our  analysis 
of  the  documentation  disclosed  that  the  test  scripts  only  tested  to  make 
sure  that  a  report  could  be  printed  for  (1)  a  specific  customer,  (2)  a 
consecutive  range  of  customers  (e.g.,  customers  1  through  10),  (3)  federal 
and  nonfederal  customers  (the  exact  condition  tested  was  not  specified 
since  the  user  was  only  required  to  pick  one  of  the  conditions),  and 
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(4)  receivables  associated  with  a  specified  general  ledger  account.  While 
this  testing  addressed  the  requirements  that  the  report  be  printed  using 
these  conditions,  the  scripts  did  not  provide  evidence  that  other 
requirements  were  tested  and  the  data  presented  were  valid.  For  example, 
the  report  was  expected  to  put  receivables  into  11  different  age  categories 
(for  example  0  to  30  days,  31  to  60  days,  etc.);  however,  no  evidence  was 
provided  to  show  that  the  balances  shown  on  these  reports  (1)  were 
consistent  with  known  results  and  (2)  the  accounts  selected  provided 
adequate  coverage  of  the  ranges  specified.  The  latter  is  especially 
important  since  two  of  the  ranges  overlap.  LMP  and  CSC  officials  agreed 
that  the  documentation  did  not  always  provide  the  level  of  detail  that  was 
needed  to  (1)  document  which  requirements  contained  in  applicable 
requirement  documents  were  being  tested  and  (2)  ensure  that  all  of  the 
specific  tests  that  were  required  to  provide  reasonable  assurance  that  the 
application  worked  as  expected  were  completed. 

Furthermore,  we  were  informed  that  the  system  testing  was  conducted  by 
the  developers  and  subject  matter  experts.  Using  developers  and  subject 
matter  experts,  and  depending  on  those  individuals’  comprehensive  body 
of  knowledge  as  the  foundation  for  a  testing  effort,  carries  significant  risks 
since  this  is  not  a  recognized  best  practice.  A  basic  testing  principle  is  that 
a  developer  should  not  test  his  or  her  own  work,  nor  should  a 
development  organization  test  its  own  work.  These  testing  principles  are 
based  on  the  concept  that  testing  is  the  process  of  executing  a  program 
with  the  intent  of  finding  errors.  Testing  is  normally  considered  a 
“destructive”  process  while  the  development  activities  are  normally 
considered  a  “constructive”  progress.  It  is  very  difficult  for  a  developer  to 
change  the  perspective  required  to  successfully  develop  a  program  into 
the  perspective  necessary  to  adequately  test  a  process.5  This  does  not 
mean  that  the  developers  are  not  involved  in  the  testing  process  since  a 
best  practice  is  to  have  them  responsible  for  testing  the  code  they  develop 
to  ensure  that  it  is  ready  for  the  next  stage  of  testing.6 

Further,  as  noted  in  our  May  2004  report,7  our  analysis  of  LMP’s  December 
2003  and  January  2004  project  status  reports  identified  numerous 


5Glenford  J.  Myers,  The  Art  of  Software  Testing  (New  York,  N.Y.:  John  Wiley  &  Sons,  Inc., 
1979). 

6Rex  Black,  Critical  Testing  Processes:  Plan,  Prepare,  Perform,  Perfect  (Boston,  Mass.: 
Addison-Wesley,  2004). 

7GAO-04-615. 


Page  57 


GAO-07-860  Army  Asset  Visibility 


Appendix  IV:  LMP  Problems  Continue 


Improved  Testing  Process 
Planned 


instances  in  which  the  Army  continued  to  experience  problems  with  the 
accuracy  of  data  related  to  budgeting;  workload  planning  and  forecasting 
and  depot  maintenance  operations;  and  accounting  records  such  as 
customer  orders,  purchase  orders  and  requisitions,  obligations,  and 
disbursements.  DFAS  and  Army  officials  acknowledged  that  these 
problems  were  attributable  to  relying  on  subject  matter  experts  to  develop 
tests  for  their  respective  functional  areas,  and  not  performing  end-to-end 
testing  across  the  various  functional  areas. 

After  discussing  our  concerns  with  the  LMP  program  office  and  CSC 
officials  in  February  2007,  the  LMP  program  office  requested  the 
verification  and  validation  (V&V)  contractor  to  perform  an  assessment  of 
the  LMP  testing  process.  According  to  information  provided  by  the  LMP 
program  office  and  the  V&V  contractor,  in  March  2007,  the  review  found 
that  the  LMP  testing  process  was  adequately  defined  in  the  planning 
documentation  and  noted  that  the  LMP  testing  philosophy  was  governed 
by  the  concepts  that  (1)  the  change  should  operate  as  intended,  (2)  the 
change  should  not  be  harmful  to  the  existing  functional  environment, 

(3)  testing  should  be  practical,  and  (4)  the  testing  efforts  should  be  cost 
efficient.  The  V&V  contractor  analysis  of  the  LMP  testing  process  also 
found  that  the  testing  documentation — which  was  the  same 
documentation  that  we  analyzed  did  not  support  a  clear  understanding  of 
the  (1)  requirement  being  tested  and  (2)  tests  used  to  determine  whether  a 
requirement  was  adequately  implemented.  According  to  the  V&V 
contractor,  these  key  areas  need  to  be  addressed.  The  V&V  contractor 
assessment  substantiates  our  analysis  of  the  LMP  testing  process. 


LMP  program  office  officials  and  the  V&V  contractor  agreed  that  the 
testing  process  currently  being  utilized  could  be  improved.  They  also 
noted  that  the  LMP  program  had  begun  implementing  an  initiative  to 
improve  the  testing  program  that  should  address  the  weaknesses  we 
identified.  The  following  outlines  two  key  components  of  LMP’s  improved 
testing  efforts. 

Establishment  of  an  independent  test  group.  This  is  a  critical  step 
and,  if  effectively  implemented,  can  serve  as  a  strong  foundation  for 
building  an  effective  testing  process.  An  independent  testing  group  is  a 
best  practice. 

Development  and  implementation  of  an  automated  testing 
program.  CSC  is  developing  an  automated  testing  process  which  was 
expected  to  reduce  the  testing  burden  and  eliminate  user  errors  in  the 
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testing  process.  For  example,  the  user  may  not  detect  that  the  application 
did  not  provide  the  expected  result  or  record  an  error  when  the 
application  did  operate  as  expected.  These  are  commonly  referred  to  as 
testing  defects  and  are  to  be  expected  when  manual  testing  activities  are 
conducted. 

LMP  program  officials  are  of  the  opinion  that  these  two  actions  will  help 
address  the  problems  of  adequately  linking  the  requirements  to  the  testing 
and  documenting  the  testing  that  is  actually  performed.  These  actions,  if 
effectively  implemented,  are  a  significant  step  forward  and  can  help 
reduce  the  risks  associated  with  testing  to  acceptable  levels.  However,  the 
effective  implementation  will  have  to  overcome  a  number  of  past 
problems,  including  ensuring  that  the  tests  provide  adequate  coverage. 
While  developing  automated  tests  can  improve  the  reliability  of  the  testing 
efforts  and  help  facilitate  an  effective  regression  testing  program,  the  key 
is  whether  the  automated  tests  are  testing  the  correct  items.  In  our 
discussions  with  CSC,  we  were  informed  that  the  existing  test  scripts  were 
the  basis  for  developing  the  new  automated  test.  Whether  this  approach 
will  provide  an  effective  testing  process  is  yet  to  be  seen.  As  we  noted 
above,  the  test  scripts  that  will  be  used  as  the  basis  for  developing  the 
automated  test  scripts  do  not  provide  the  necessary  link  to  the 
requirements  that  are  being  tested  or  provide  the  detailed  information 
necessary  to  write  the  detailed  test  scripts  called  for  in  the  automated 
process. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday,  GAO  posts 
newly  released  reports,  testimony,  and  correspondence  on  its  Web  site.  To 
have  GAO  e-mail  you  a  list  of  newly  posted  products  every  afternoon,  go 
to  www.gao.gov  and  select  “Subscribe  to  Updates.” 
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The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each. 

A  check  or  money  order  should  be  made  out  to  the  Superintendent  of 
Documents.  GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or 
more  copies  mailed  to  a  single  address  are  discounted  25  percent.  Orders 
should  be  sent  to: 

U.S.  Government  Accountability  Office 

441  G  Street  NW,  Room  LM 

Washington,  D.C.  20548 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Gloria  Jarmon,  Managing  Director,  JarmonG@gao.gov  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  D.C.  20548 

Public  Affairs 


Paul  Anderson,  Managing  Director,  AndersonPl@gao.gov  (202)  512-4800 
U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  D.C.  20548 


PRINTED  ON  RECYCLED  PAPER 


